diff options
Diffstat (limited to 'vendor/github.com/spf13/afero/README.md')
-rw-r--r-- | vendor/github.com/spf13/afero/README.md | 452 |
1 files changed, 452 insertions, 0 deletions
diff --git a/vendor/github.com/spf13/afero/README.md b/vendor/github.com/spf13/afero/README.md new file mode 100644 index 0000000..0c9b04b --- /dev/null +++ b/vendor/github.com/spf13/afero/README.md | |||
@@ -0,0 +1,452 @@ | |||
1 | ![afero logo-sm](https://cloud.githubusercontent.com/assets/173412/11490338/d50e16dc-97a5-11e5-8b12-019a300d0fcb.png) | ||
2 | |||
3 | A FileSystem Abstraction System for Go | ||
4 | |||
5 | [![Build Status](https://travis-ci.org/spf13/afero.svg)](https://travis-ci.org/spf13/afero) [![Build status](https://ci.appveyor.com/api/projects/status/github/spf13/afero?branch=master&svg=true)](https://ci.appveyor.com/project/spf13/afero) [![GoDoc](https://godoc.org/github.com/spf13/afero?status.svg)](https://godoc.org/github.com/spf13/afero) [![Join the chat at https://gitter.im/spf13/afero](https://badges.gitter.im/Dev%20Chat.svg)](https://gitter.im/spf13/afero?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) | ||
6 | |||
7 | # Overview | ||
8 | |||
9 | Afero is an filesystem framework providing a simple, uniform and universal API | ||
10 | interacting with any filesystem, as an abstraction layer providing interfaces, | ||
11 | types and methods. Afero has an exceptionally clean interface and simple design | ||
12 | without needless constructors or initialization methods. | ||
13 | |||
14 | Afero is also a library providing a base set of interoperable backend | ||
15 | filesystems that make it easy to work with afero while retaining all the power | ||
16 | and benefit of the os and ioutil packages. | ||
17 | |||
18 | Afero provides significant improvements over using the os package alone, most | ||
19 | notably the ability to create mock and testing filesystems without relying on the disk. | ||
20 | |||
21 | It is suitable for use in a any situation where you would consider using the OS | ||
22 | package as it provides an additional abstraction that makes it easy to use a | ||
23 | memory backed file system during testing. It also adds support for the http | ||
24 | filesystem for full interoperability. | ||
25 | |||
26 | |||
27 | ## Afero Features | ||
28 | |||
29 | * A single consistent API for accessing a variety of filesystems | ||
30 | * Interoperation between a variety of file system types | ||
31 | * A set of interfaces to encourage and enforce interoperability between backends | ||
32 | * An atomic cross platform memory backed file system | ||
33 | * Support for compositional (union) file systems by combining multiple file systems acting as one | ||
34 | * Specialized backends which modify existing filesystems (Read Only, Regexp filtered) | ||
35 | * A set of utility functions ported from io, ioutil & hugo to be afero aware | ||
36 | |||
37 | |||
38 | # Using Afero | ||
39 | |||
40 | Afero is easy to use and easier to adopt. | ||
41 | |||
42 | A few different ways you could use Afero: | ||
43 | |||
44 | * Use the interfaces alone to define you own file system. | ||
45 | * Wrap for the OS packages. | ||
46 | * Define different filesystems for different parts of your application. | ||
47 | * Use Afero for mock filesystems while testing | ||
48 | |||
49 | ## Step 1: Install Afero | ||
50 | |||
51 | First use go get to install the latest version of the library. | ||
52 | |||
53 | $ go get github.com/spf13/afero | ||
54 | |||
55 | Next include Afero in your application. | ||
56 | ```go | ||
57 | import "github.com/spf13/afero" | ||
58 | ``` | ||
59 | |||
60 | ## Step 2: Declare a backend | ||
61 | |||
62 | First define a package variable and set it to a pointer to a filesystem. | ||
63 | ```go | ||
64 | var AppFs = afero.NewMemMapFs() | ||
65 | |||
66 | or | ||
67 | |||
68 | var AppFs = afero.NewOsFs() | ||
69 | ``` | ||
70 | It is important to note that if you repeat the composite literal you | ||
71 | will be using a completely new and isolated filesystem. In the case of | ||
72 | OsFs it will still use the same underlying filesystem but will reduce | ||
73 | the ability to drop in other filesystems as desired. | ||
74 | |||
75 | ## Step 3: Use it like you would the OS package | ||
76 | |||
77 | Throughout your application use any function and method like you normally | ||
78 | would. | ||
79 | |||
80 | So if my application before had: | ||
81 | ```go | ||
82 | os.Open('/tmp/foo') | ||
83 | ``` | ||
84 | We would replace it with: | ||
85 | ```go | ||
86 | AppFs.Open('/tmp/foo') | ||
87 | ``` | ||
88 | |||
89 | `AppFs` being the variable we defined above. | ||
90 | |||
91 | |||
92 | ## List of all available functions | ||
93 | |||
94 | File System Methods Available: | ||
95 | ```go | ||
96 | Chmod(name string, mode os.FileMode) : error | ||
97 | Chtimes(name string, atime time.Time, mtime time.Time) : error | ||
98 | Create(name string) : File, error | ||
99 | Mkdir(name string, perm os.FileMode) : error | ||
100 | MkdirAll(path string, perm os.FileMode) : error | ||
101 | Name() : string | ||
102 | Open(name string) : File, error | ||
103 | OpenFile(name string, flag int, perm os.FileMode) : File, error | ||
104 | Remove(name string) : error | ||
105 | RemoveAll(path string) : error | ||
106 | Rename(oldname, newname string) : error | ||
107 | Stat(name string) : os.FileInfo, error | ||
108 | ``` | ||
109 | File Interfaces and Methods Available: | ||
110 | ```go | ||
111 | io.Closer | ||
112 | io.Reader | ||
113 | io.ReaderAt | ||
114 | io.Seeker | ||
115 | io.Writer | ||
116 | io.WriterAt | ||
117 | |||
118 | Name() : string | ||
119 | Readdir(count int) : []os.FileInfo, error | ||
120 | Readdirnames(n int) : []string, error | ||
121 | Stat() : os.FileInfo, error | ||
122 | Sync() : error | ||
123 | Truncate(size int64) : error | ||
124 | WriteString(s string) : ret int, err error | ||
125 | ``` | ||
126 | In some applications it may make sense to define a new package that | ||
127 | simply exports the file system variable for easy access from anywhere. | ||
128 | |||
129 | ## Using Afero's utility functions | ||
130 | |||
131 | Afero provides a set of functions to make it easier to use the underlying file systems. | ||
132 | These functions have been primarily ported from io & ioutil with some developed for Hugo. | ||
133 | |||
134 | The afero utilities support all afero compatible backends. | ||
135 | |||
136 | The list of utilities includes: | ||
137 | |||
138 | ```go | ||
139 | DirExists(path string) (bool, error) | ||
140 | Exists(path string) (bool, error) | ||
141 | FileContainsBytes(filename string, subslice []byte) (bool, error) | ||
142 | GetTempDir(subPath string) string | ||
143 | IsDir(path string) (bool, error) | ||
144 | IsEmpty(path string) (bool, error) | ||
145 | ReadDir(dirname string) ([]os.FileInfo, error) | ||
146 | ReadFile(filename string) ([]byte, error) | ||
147 | SafeWriteReader(path string, r io.Reader) (err error) | ||
148 | TempDir(dir, prefix string) (name string, err error) | ||
149 | TempFile(dir, prefix string) (f File, err error) | ||
150 | Walk(root string, walkFn filepath.WalkFunc) error | ||
151 | WriteFile(filename string, data []byte, perm os.FileMode) error | ||
152 | WriteReader(path string, r io.Reader) (err error) | ||
153 | ``` | ||
154 | For a complete list see [Afero's GoDoc](https://godoc.org/github.com/spf13/afero) | ||
155 | |||
156 | They are available under two different approaches to use. You can either call | ||
157 | them directly where the first parameter of each function will be the file | ||
158 | system, or you can declare a new `Afero`, a custom type used to bind these | ||
159 | functions as methods to a given filesystem. | ||
160 | |||
161 | ### Calling utilities directly | ||
162 | |||
163 | ```go | ||
164 | fs := new(afero.MemMapFs) | ||
165 | f, err := afero.TempFile(fs,"", "ioutil-test") | ||
166 | |||
167 | ``` | ||
168 | |||
169 | ### Calling via Afero | ||
170 | |||
171 | ```go | ||
172 | fs := afero.NewMemMapFs() | ||
173 | afs := &afero.Afero{Fs: fs} | ||
174 | f, err := afs.TempFile("", "ioutil-test") | ||
175 | ``` | ||
176 | |||
177 | ## Using Afero for Testing | ||
178 | |||
179 | There is a large benefit to using a mock filesystem for testing. It has a | ||
180 | completely blank state every time it is initialized and can be easily | ||
181 | reproducible regardless of OS. You could create files to your heart’s content | ||
182 | and the file access would be fast while also saving you from all the annoying | ||
183 | issues with deleting temporary files, Windows file locking, etc. The MemMapFs | ||
184 | backend is perfect for testing. | ||
185 | |||
186 | * Much faster than performing I/O operations on disk | ||
187 | * Avoid security issues and permissions | ||
188 | * Far more control. 'rm -rf /' with confidence | ||
189 | * Test setup is far more easier to do | ||
190 | * No test cleanup needed | ||
191 | |||
192 | One way to accomplish this is to define a variable as mentioned above. | ||
193 | In your application this will be set to afero.NewOsFs() during testing you | ||
194 | can set it to afero.NewMemMapFs(). | ||
195 | |||
196 | It wouldn't be uncommon to have each test initialize a blank slate memory | ||
197 | backend. To do this I would define my `appFS = afero.NewOsFs()` somewhere | ||
198 | appropriate in my application code. This approach ensures that Tests are order | ||
199 | independent, with no test relying on the state left by an earlier test. | ||
200 | |||
201 | Then in my tests I would initialize a new MemMapFs for each test: | ||
202 | ```go | ||
203 | func TestExist(t *testing.T) { | ||
204 | appFS := afero.NewMemMapFs() | ||
205 | // create test files and directories | ||
206 | appFS.MkdirAll("src/a", 0755) | ||
207 | afero.WriteFile(appFS, "src/a/b", []byte("file b"), 0644) | ||
208 | afero.WriteFile(appFS, "src/c", []byte("file c"), 0644) | ||
209 | name := "src/c" | ||
210 | _, err := appFS.Stat(name) | ||
211 | if os.IsNotExist(err) { | ||
212 | t.Errorf("file \"%s\" does not exist.\n", name) | ||
213 | } | ||
214 | } | ||
215 | ``` | ||
216 | |||
217 | # Available Backends | ||
218 | |||
219 | ## Operating System Native | ||
220 | |||
221 | ### OsFs | ||
222 | |||
223 | The first is simply a wrapper around the native OS calls. This makes it | ||
224 | very easy to use as all of the calls are the same as the existing OS | ||
225 | calls. It also makes it trivial to have your code use the OS during | ||
226 | operation and a mock filesystem during testing or as needed. | ||
227 | |||
228 | ```go | ||
229 | appfs := afero.NewOsFs() | ||
230 | appfs.MkdirAll("src/a", 0755)) | ||
231 | ``` | ||
232 | |||
233 | ## Memory Backed Storage | ||
234 | |||
235 | ### MemMapFs | ||
236 | |||
237 | Afero also provides a fully atomic memory backed filesystem perfect for use in | ||
238 | mocking and to speed up unnecessary disk io when persistence isn’t | ||
239 | necessary. It is fully concurrent and will work within go routines | ||
240 | safely. | ||
241 | |||
242 | ```go | ||
243 | mm := afero.NewMemMapFs() | ||
244 | mm.MkdirAll("src/a", 0755)) | ||
245 | ``` | ||
246 | |||
247 | #### InMemoryFile | ||
248 | |||
249 | As part of MemMapFs, Afero also provides an atomic, fully concurrent memory | ||
250 | backed file implementation. This can be used in other memory backed file | ||
251 | systems with ease. Plans are to add a radix tree memory stored file | ||
252 | system using InMemoryFile. | ||
253 | |||
254 | ## Network Interfaces | ||
255 | |||
256 | ### SftpFs | ||
257 | |||
258 | Afero has experimental support for secure file transfer protocol (sftp). Which can | ||
259 | be used to perform file operations over a encrypted channel. | ||
260 | |||
261 | ## Filtering Backends | ||
262 | |||
263 | ### BasePathFs | ||
264 | |||
265 | The BasePathFs restricts all operations to a given path within an Fs. | ||
266 | The given file name to the operations on this Fs will be prepended with | ||
267 | the base path before calling the source Fs. | ||
268 | |||
269 | ```go | ||
270 | bp := afero.NewBasePathFs(afero.NewOsFs(), "/base/path") | ||
271 | ``` | ||
272 | |||
273 | ### ReadOnlyFs | ||
274 | |||
275 | A thin wrapper around the source Fs providing a read only view. | ||
276 | |||
277 | ```go | ||
278 | fs := afero.NewReadOnlyFs(afero.NewOsFs()) | ||
279 | _, err := fs.Create("/file.txt") | ||
280 | // err = syscall.EPERM | ||
281 | ``` | ||
282 | |||
283 | # RegexpFs | ||
284 | |||
285 | A filtered view on file names, any file NOT matching | ||
286 | the passed regexp will be treated as non-existing. | ||
287 | Files not matching the regexp provided will not be created. | ||
288 | Directories are not filtered. | ||
289 | |||
290 | ```go | ||
291 | fs := afero.NewRegexpFs(afero.NewMemMapFs(), regexp.MustCompile(`\.txt$`)) | ||
292 | _, err := fs.Create("/file.html") | ||
293 | // err = syscall.ENOENT | ||
294 | ``` | ||
295 | |||
296 | ### HttpFs | ||
297 | |||
298 | Afero provides an http compatible backend which can wrap any of the existing | ||
299 | backends. | ||
300 | |||
301 | The Http package requires a slightly specific version of Open which | ||
302 | returns an http.File type. | ||
303 | |||
304 | Afero provides an httpFs file system which satisfies this requirement. | ||
305 | Any Afero FileSystem can be used as an httpFs. | ||
306 | |||
307 | ```go | ||
308 | httpFs := afero.NewHttpFs(<ExistingFS>) | ||
309 | fileserver := http.FileServer(httpFs.Dir(<PATH>))) | ||
310 | http.Handle("/", fileserver) | ||
311 | ``` | ||
312 | |||
313 | ## Composite Backends | ||
314 | |||
315 | Afero provides the ability have two filesystems (or more) act as a single | ||
316 | file system. | ||
317 | |||
318 | ### CacheOnReadFs | ||
319 | |||
320 | The CacheOnReadFs will lazily make copies of any accessed files from the base | ||
321 | layer into the overlay. Subsequent reads will be pulled from the overlay | ||
322 | directly permitting the request is within the cache duration of when it was | ||
323 | created in the overlay. | ||
324 | |||
325 | If the base filesystem is writeable, any changes to files will be | ||
326 | done first to the base, then to the overlay layer. Write calls to open file | ||
327 | handles like `Write()` or `Truncate()` to the overlay first. | ||
328 | |||
329 | To writing files to the overlay only, you can use the overlay Fs directly (not | ||
330 | via the union Fs). | ||
331 | |||
332 | Cache files in the layer for the given time.Duration, a cache duration of 0 | ||
333 | means "forever" meaning the file will not be re-requested from the base ever. | ||
334 | |||
335 | A read-only base will make the overlay also read-only but still copy files | ||
336 | from the base to the overlay when they're not present (or outdated) in the | ||
337 | caching layer. | ||
338 | |||
339 | ```go | ||
340 | base := afero.NewOsFs() | ||
341 | layer := afero.NewMemMapFs() | ||
342 | ufs := afero.NewCacheOnReadFs(base, layer, 100 * time.Second) | ||
343 | ``` | ||
344 | |||
345 | ### CopyOnWriteFs() | ||
346 | |||
347 | The CopyOnWriteFs is a read only base file system with a potentially | ||
348 | writeable layer on top. | ||
349 | |||
350 | Read operations will first look in the overlay and if not found there, will | ||
351 | serve the file from the base. | ||
352 | |||
353 | Changes to the file system will only be made in the overlay. | ||
354 | |||
355 | Any attempt to modify a file found only in the base will copy the file to the | ||
356 | overlay layer before modification (including opening a file with a writable | ||
357 | handle). | ||
358 | |||
359 | Removing and Renaming files present only in the base layer is not currently | ||
360 | permitted. If a file is present in the base layer and the overlay, only the | ||
361 | overlay will be removed/renamed. | ||
362 | |||
363 | ```go | ||
364 | base := afero.NewOsFs() | ||
365 | roBase := afero.NewReadOnlyFs(base) | ||
366 | ufs := afero.NewCopyOnWriteFs(roBase, afero.NewMemMapFs()) | ||
367 | |||
368 | fh, _ = ufs.Create("/home/test/file2.txt") | ||
369 | fh.WriteString("This is a test") | ||
370 | fh.Close() | ||
371 | ``` | ||
372 | |||
373 | In this example all write operations will only occur in memory (MemMapFs) | ||
374 | leaving the base filesystem (OsFs) untouched. | ||
375 | |||
376 | |||
377 | ## Desired/possible backends | ||
378 | |||
379 | The following is a short list of possible backends we hope someone will | ||
380 | implement: | ||
381 | |||
382 | * SSH | ||
383 | * ZIP | ||
384 | * TAR | ||
385 | * S3 | ||
386 | |||
387 | # About the project | ||
388 | |||
389 | ## What's in the name | ||
390 | |||
391 | Afero comes from the latin roots Ad-Facere. | ||
392 | |||
393 | **"Ad"** is a prefix meaning "to". | ||
394 | |||
395 | **"Facere"** is a form of the root "faciō" making "make or do". | ||
396 | |||
397 | The literal meaning of afero is "to make" or "to do" which seems very fitting | ||
398 | for a library that allows one to make files and directories and do things with them. | ||
399 | |||
400 | The English word that shares the same roots as Afero is "affair". Affair shares | ||
401 | the same concept but as a noun it means "something that is made or done" or "an | ||
402 | object of a particular type". | ||
403 | |||
404 | It's also nice that unlike some of my other libraries (hugo, cobra, viper) it | ||
405 | Googles very well. | ||
406 | |||
407 | ## Release Notes | ||
408 | |||
409 | * **0.10.0** 2015.12.10 | ||
410 | * Full compatibility with Windows | ||
411 | * Introduction of afero utilities | ||
412 | * Test suite rewritten to work cross platform | ||
413 | * Normalize paths for MemMapFs | ||
414 | * Adding Sync to the file interface | ||
415 | * **Breaking Change** Walk and ReadDir have changed parameter order | ||
416 | * Moving types used by MemMapFs to a subpackage | ||
417 | * General bugfixes and improvements | ||
418 | * **0.9.0** 2015.11.05 | ||
419 | * New Walk function similar to filepath.Walk | ||
420 | * MemMapFs.OpenFile handles O_CREATE, O_APPEND, O_TRUNC | ||
421 | * MemMapFs.Remove now really deletes the file | ||
422 | * InMemoryFile.Readdir and Readdirnames work correctly | ||
423 | * InMemoryFile functions lock it for concurrent access | ||
424 | * Test suite improvements | ||
425 | * **0.8.0** 2014.10.28 | ||
426 | * First public version | ||
427 | * Interfaces feel ready for people to build using | ||
428 | * Interfaces satisfy all known uses | ||
429 | * MemMapFs passes the majority of the OS test suite | ||
430 | * OsFs passes the majority of the OS test suite | ||
431 | |||
432 | ## Contributing | ||
433 | |||
434 | 1. Fork it | ||
435 | 2. Create your feature branch (`git checkout -b my-new-feature`) | ||
436 | 3. Commit your changes (`git commit -am 'Add some feature'`) | ||
437 | 4. Push to the branch (`git push origin my-new-feature`) | ||
438 | 5. Create new Pull Request | ||
439 | |||
440 | ## Contributors | ||
441 | |||
442 | Names in no particular order: | ||
443 | |||
444 | * [spf13](https://github.com/spf13) | ||
445 | * [jaqx0r](https://github.com/jaqx0r) | ||
446 | * [mbertschler](https://github.com/mbertschler) | ||
447 | * [xor-gate](https://github.com/xor-gate) | ||
448 | |||
449 | ## License | ||
450 | |||
451 | Afero is released under the Apache 2.0 license. See | ||
452 | [LICENSE.txt](https://github.com/spf13/afero/blob/master/LICENSE.txt) | ||