aboutsummaryrefslogtreecommitdiffhomepage
path: root/vendor/github.com/ulikunitz/xz/TODO.md
diff options
context:
space:
mode:
Diffstat (limited to 'vendor/github.com/ulikunitz/xz/TODO.md')
-rw-r--r--vendor/github.com/ulikunitz/xz/TODO.md315
1 files changed, 315 insertions, 0 deletions
diff --git a/vendor/github.com/ulikunitz/xz/TODO.md b/vendor/github.com/ulikunitz/xz/TODO.md
new file mode 100644
index 0000000..7b34c0c
--- /dev/null
+++ b/vendor/github.com/ulikunitz/xz/TODO.md
@@ -0,0 +1,315 @@
1# TODO list
2
3## Release v0.6
4
51. Review encoder and check for lzma improvements under xz.
62. Fix binary tree matcher.
73. Compare compression ratio with xz tool using comparable parameters
8 and optimize parameters
94. Do some optimizations
10 - rename operation action and make it a simple type of size 8
11 - make maxMatches, wordSize parameters
12 - stop searching after a certain length is found (parameter sweetLen)
13
14## Release v0.7
15
161. Optimize code
172. Do statistical analysis to get linear presets.
183. Test sync.Pool compatability for xz and lzma Writer and Reader
193. Fuzz optimized code.
20
21## Release v0.8
22
231. Support parallel go routines for writing and reading xz files.
242. Support a ReaderAt interface for xz files with small block sizes.
253. Improve compatibility between gxz and xz
264. Provide manual page for gxz
27
28## Release v0.9
29
301. Improve documentation
312. Fuzz again
32
33## Release v1.0
34
351. Full functioning gxz
362. Add godoc URL to README.md (godoc.org)
373. Resolve all issues.
384. Define release candidates.
395. Public announcement.
40
41## Package lzma
42
43### Release v0.6
44
45- Rewrite Encoder into a simple greedy one-op-at-a-time encoder
46 including
47 + simple scan at the dictionary head for the same byte
48 + use the killer byte (requiring matches to get longer, the first
49 test should be the byte that would make the match longer)
50
51
52## Optimizations
53
54- There may be a lot of false sharing in lzma.State; check whether this
55 can be improved by reorganizing the internal structure of it.
56- Check whether batching encoding and decoding improves speed.
57
58### DAG optimizations
59
60- Use full buffer to create minimal bit-length above range encoder.
61- Might be too slow (see v0.4)
62
63### Different match finders
64
65- hashes with 2, 3 characters additional to 4 characters
66- binary trees with 2-7 characters (uint64 as key, use uint32 as
67 pointers into a an array)
68- rb-trees with 2-7 characters (uint64 as key, use uint32 as pointers
69 into an array with bit-steeling for the colors)
70
71## Release Procedure
72
73- execute goch -l for all packages; probably with lower param like 0.5.
74- check orthography with gospell
75- Write release notes in doc/relnotes.
76- Update README.md
77- xb copyright . in xz directory to ensure all new files have Copyright
78 header
79- VERSION=<version> go generate github.com/ulikunitz/xz/... to update
80 version files
81- Execute test for Linux/amd64, Linux/x86 and Windows/amd64.
82- Update TODO.md - write short log entry
83- git checkout master && git merge dev
84- git tag -a <version>
85- git push
86
87## Log
88
89### 2017-06-05
90
91Release v0.5.4 fixes issues #15 of another problem with the padding size
92check for the xz block header. I removed the check completely.
93
94### 2017-02-15
95
96Release v0.5.3 fixes issue #12 regarding the decompression of an empty
97XZ stream. Many thanks to Tomasz Kłak, who reported the issue.
98
99### 2016-12-02
100
101Release v0.5.2 became necessary to allow the decoding of xz files with
1024-byte padding in the block header. Many thanks to Greg, who reported
103the issue.
104
105### 2016-07-23
106
107Release v0.5.1 became necessary to fix problems with 32-bit platforms.
108Many thanks to Bruno Brigas, who reported the issue.
109
110### 2016-07-04
111
112Release v0.5 provides improvements to the compressor and provides support for
113the decompression of xz files with multiple xz streams.
114
115### 2016-01-31
116
117Another compression rate increase by checking the byte at length of the
118best match first, before checking the whole prefix. This makes the
119compressor even faster. We have now a large time budget to beat the
120compression ratio of the xz tool. For enwik8 we have now over 40 seconds
121to reduce the compressed file size for another 7 MiB.
122
123### 2016-01-30
124
125I simplified the encoder. Speed and compression rate increased
126dramatically. A high compression rate affects also the decompression
127speed. The approach with the buffer and optimizing for operation
128compression rate has not been successful. Going for the maximum length
129appears to be the best approach.
130
131### 2016-01-28
132
133The release v0.4 is ready. It provides a working xz implementation,
134which is rather slow, but works and is interoperable with the xz tool.
135It is an important milestone.
136
137### 2016-01-10
138
139I have the first working implementation of an xz reader and writer. I'm
140happy about reaching this milestone.
141
142### 2015-12-02
143
144I'm now ready to implement xz because, I have a working LZMA2
145implementation. I decided today that v0.4 will use the slow encoder
146using the operations buffer to be able to go back, if I intend to do so.
147
148### 2015-10-21
149
150I have restarted the work on the library. While trying to implement
151LZMA2, I discovered that I need to resimplify the encoder and decoder
152functions. The option approach is too complicated. Using a limited byte
153writer and not caring for written bytes at all and not to try to handle
154uncompressed data simplifies the LZMA encoder and decoder much.
155Processing uncompressed data and handling limits is a feature of the
156LZMA2 format not of LZMA.
157
158I learned an interesting method from the LZO format. If the last copy is
159too far away they are moving the head one 2 bytes and not 1 byte to
160reduce processing times.
161
162### 2015-08-26
163
164I have now reimplemented the lzma package. The code is reasonably fast,
165but can still be optimized. The next step is to implement LZMA2 and then
166xz.
167
168### 2015-07-05
169
170Created release v0.3. The version is the foundation for a full xz
171implementation that is the target of v0.4.
172
173### 2015-06-11
174
175The gflag package has been developed because I couldn't use flag and
176pflag for a fully compatible support of gzip's and lzma's options. It
177seems to work now quite nicely.
178
179### 2015-06-05
180
181The overflow issue was interesting to research, however Henry S. Warren
182Jr. Hacker's Delight book was very helpful as usual and had the issue
183explained perfectly. Fefe's information on his website was based on the
184C FAQ and quite bad, because it didn't address the issue of -MININT ==
185MININT.
186
187### 2015-06-04
188
189It has been a productive day. I improved the interface of lzma.Reader
190and lzma.Writer and fixed the error handling.
191
192### 2015-06-01
193
194By computing the bit length of the LZMA operations I was able to
195improve the greedy algorithm implementation. By using an 8 MByte buffer
196the compression rate was not as good as for xz but already better then
197gzip default.
198
199Compression is currently slow, but this is something we will be able to
200improve over time.
201
202### 2015-05-26
203
204Checked the license of ogier/pflag. The binary lzmago binary should
205include the license terms for the pflag library.
206
207I added the endorsement clause as used by Google for the Go sources the
208LICENSE file.
209
210### 2015-05-22
211
212The package lzb contains now the basic implementation for creating or
213reading LZMA byte streams. It allows the support for the implementation
214of the DAG-shortest-path algorithm for the compression function.
215
216### 2015-04-23
217
218Completed yesterday the lzbase classes. I'm a little bit concerned that
219using the components may require too much code, but on the other hand
220there is a lot of flexibility.
221
222### 2015-04-22
223
224Implemented Reader and Writer during the Bayern game against Porto. The
225second half gave me enough time.
226
227### 2015-04-21
228
229While showering today morning I discovered that the design for OpEncoder
230and OpDecoder doesn't work, because encoding/decoding might depend on
231the current status of the dictionary. This is not exactly the right way
232to start the day.
233
234Therefore we need to keep the Reader and Writer design. This time around
235we simplify it by ignoring size limits. These can be added by wrappers
236around the Reader and Writer interfaces. The Parameters type isn't
237needed anymore.
238
239However I will implement a ReaderState and WriterState type to use
240static typing to ensure the right State object is combined with the
241right lzbase.Reader and lzbase.Writer.
242
243As a start I have implemented ReaderState and WriterState to ensure
244that the state for reading is only used by readers and WriterState only
245used by Writers.
246
247### 2015-04-20
248
249Today I implemented the OpDecoder and tested OpEncoder and OpDecoder.
250
251### 2015-04-08
252
253Came up with a new simplified design for lzbase. I implemented already
254the type State that replaces OpCodec.
255
256### 2015-04-06
257
258The new lzma package is now fully usable and lzmago is using it now. The
259old lzma package has been completely removed.
260
261### 2015-04-05
262
263Implemented lzma.Reader and tested it.
264
265### 2015-04-04
266
267Implemented baseReader by adapting code form lzma.Reader.
268
269### 2015-04-03
270
271The opCodec has been copied yesterday to lzma2. opCodec has a high
272number of dependencies on other files in lzma2. Therefore I had to copy
273almost all files from lzma.
274
275### 2015-03-31
276
277Removed only a TODO item.
278
279However in Francesco Campoy's presentation "Go for Javaneros
280(Javaïstes?)" is the the idea that using an embedded field E, all the
281methods of E will be defined on T. If E is an interface T satisfies E.
282
283https://talks.golang.org/2014/go4java.slide#51
284
285I have never used this, but it seems to be a cool idea.
286
287### 2015-03-30
288
289Finished the type writerDict and wrote a simple test.
290
291### 2015-03-25
292
293I started to implement the writerDict.
294
295### 2015-03-24
296
297After thinking long about the LZMA2 code and several false starts, I
298have now a plan to create a self-sufficient lzma2 package that supports
299the classic LZMA format as well as LZMA2. The core idea is to support a
300baseReader and baseWriter type that support the basic LZMA stream
301without any headers. Both types must support the reuse of dictionaries
302and the opCodec.
303
304### 2015-01-10
305
3061. Implemented simple lzmago tool
3072. Tested tool against large 4.4G file
308 - compression worked correctly; tested decompression with lzma
309 - decompression hits a full buffer condition
3103. Fixed a bug in the compressor and wrote a test for it
3114. Executed full cycle for 4.4 GB file; performance can be improved ;-)
312
313### 2015-01-11
314
315- Release v0.2 because of the working LZMA encoder and decoder