infrastructure/k8s-csi-s3
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity
217 commits 4 branches 33 tags 23 MiB
Commit graph

52 commits

Author SHA1 Message Date
Alexander Narsudinov
d3e89f164b Use atomics to make writing of results safe in removeObjectsOneByOne 
Previous implementation didn't have any synchronization mechanism for
goroutines that does the work.

There are multiple approaches to make it work correctly, let's start with
the simplest - atomics.
 2023-10-27 20:05:02 +02:00
Alexander Narsudinov
da3638eb56 Fix the goroutine variable capturing in removeObjectsOneByOne 
This patch fixes classic golang for-loop variable capturing issue that
leeds to incorrect results in goroutines.

This is fixed in latest versions of golang, but this project uses go 1.15,
so it won't work as expected by an author.
 2023-10-27 19:59:03 +02:00
x.zhou
680a649a21 Fix the wrong argument for credentials.NewStaticV4() 2023-06-09 16:10:33 +08:00
Vitaliy Filippov
ce0d5e7704 Delete objects in parallel when deleting auto-created volumes 2021-07-27 13:10:45 +03:00
Vitaliy Filippov
55013c94bc Add 3 FIXMEs 2021-07-26 21:11:32 +03:00
Vitaliy Filippov
2089c40001 Do not store additional metadata in .metadata.json in the bucket itself 2021-07-26 21:11:32 +03:00
Vitaliy Filippov
9562699f1b Remove additional FSPath prefix 2021-07-19 15:41:45 +03:00
Vitaliy Filippov
d78d476d6d Add mount options support 2021-07-16 16:12:57 +03:00
boxjan
8449c1e2e9 Although get a EOF error, we should still know it 2021-04-09 21:45:52 +08:00
boxjan
fa9ef78362 fix problem: log format; fs meta; 2021-04-08 22:35:31 +08:00
boxjan
7b5a3f6f4d fix problem 
* rclone use 1.54.1
* add EnsureFSMetaExist func
* fix removeObjects have err check
* add removeObjectsOneByOne, when removeObjects failed, will try to use it
* fix repeat in yaml
 2021-04-08 00:06:56 +08:00
Cyrill Troxler
b11261ebe7 Only delete bucket if the volume is the alone user 2021-04-06 19:24:09 +02:00
Cyrill Troxler
26cb1d95e8 Use volume ID as a prefix if the bucket is fixed in the storage class 
With this, each volume will get its own prefix within the bucket if it
is configured in the storage class. This also ensures backwards
compatibility with older volumes that have been created in earlier
versions of csi-s3.
 2021-04-05 15:07:16 +02:00
Cyrill Troxler
7528b3297c Rename packages 
Create separate packages for driver, s3 client and mounters.
 2021-04-05 12:10:20 +02:00
Lin Yiming
fbc265dff7 If bucket is not created by csi-s3, it will not be deleted by csi-s3 automatically 2021-01-22 10:15:21 +08:00
Lin Yiming
fae1210aaf Do not create fs prefix for existing bucket 2021-01-22 10:15:21 +08:00
Lin Yiming
7e1842c274 mount existed bucket 2021-01-22 10:15:21 +08:00
Lin Yiming
a68fc3379e upgrade minio-go library to v7 2021-01-22 10:15:21 +08:00
Carl Lange
95962c2412
Use NewWithRegion not minio's region resolution 2019-06-30 16:32:59 +01:00
Cyrill Troxler
fb6584d7e4 Release new version 2019-05-18 11:55:37 +02:00
Cyrill Troxler
e133baaba1 Clean up old references to encryptionKey 2019-05-16 18:15:45 +02:00
Cyrill Troxler
f80104f215 Remove s3ql 
s3ql does not work very well in a dynamic environment like k8s.
Also as it needs a ton of dependencies just to get it built makes
it hard to maintain.
 2019-05-15 21:06:28 +02:00
Cyrill Troxler
76fc704c31 Do not use sigv2 with s3fs 2019-05-15 21:00:16 +02:00
Cyrill Troxler
cc38b81678 Add log message when mounting 2019-05-15 21:00:16 +02:00
Cyrill Troxler
152eeb1d14 Update README and deployment, split up dockerfile 2019-05-15 21:00:16 +02:00
Cyrill Troxler
c6b3b5199d Upgrade csi-spec, fix tests 2019-05-15 21:00:16 +02:00
Cyrill Troxler
4e8bbf0126 Supply credentials using volume secrets instead of cli config 2019-05-15 21:00:02 +02:00
Cyrill Troxler
ea4022e9c6
Add rclone mounter (#15) 2019-03-07 20:27:02 +01:00
Cyrill Troxler
95d7e48328
Revert capabilites to only allow RWO (#12) 2019-02-28 21:09:56 +01:00
Lennard
83723f4be0 Feature/update to csi 1.0 (#9) 
* updated all image to v1.0.1 csi

* Updated golang code to csi 1.0 compatible names

* added comment
 2019-02-28 19:34:03 +01:00
Cyrill Troxler
58912e4734 Sanitize the volumeID before creation 
* Bucket names cannot contain upper case letters
* Bucket names cannot be longer than 63 chars
 2019-02-06 20:02:28 +01:00
Cyrill Troxler
728ef7d7e7 Fix s3backer with AWS 
If we set baseURL with AWS s3backer will fail to authenticate.
 2019-01-19 14:43:54 +01:00
Cyrill Troxler
709162c21b Set correct vendor version and display it on launch 2019-01-19 14:40:58 +01:00
Cyrill Troxler
d8f966a3de Improve bucket error messages 2019-01-19 14:39:33 +01:00
Cyrill Troxler
9291d09baa
Add ci pipeline with GitLab (#3) 
* Add automated testing
* Create loop device while staging s3backer
 2018-08-03 20:30:46 +02:00
Cyrill Troxler
8a31ac89c9 Improve bucket error messages 2018-07-29 16:19:26 +02:00
Cyrill Troxler
df04f577d2 Rename to just csi-s3 2018-07-29 14:49:35 +02:00
Cyrill Troxler
c7abe28ade Allow configuring mounter via storageclass parameter 2018-07-29 14:36:13 +02:00
Cyrill Troxler
e46440d1fa Add fs prefix (directory) 
This ensures the fs root is clean and does not mess with
the metadata. Also in the future this will allow for multiple
filesystems to be created in one bucket.
 2018-07-27 21:37:32 +02:00
Cyrill Troxler
1caf469966 Implement a metadata file and correct sizing 
As the controller does not mount/create the fs we have to store the capacity somewhere so the node knows about it.
 2018-07-27 13:28:52 +02:00
Cyrill Troxler
0010066fe3 Implement stage/unstage 
This helps the reliability of s3backer as the fuse mount
is done on NodeStageVolume and only once per volume per
node.
 2018-07-26 22:43:51 +02:00
Cyrill Troxler
1fe218a568 Format using nodeserver, not controller 2018-07-26 20:13:40 +02:00
Cyrill Troxler
f8bd74afb9 Fix log output 2018-07-23 20:58:25 +02:00
Cyrill Troxler
a468d955bc Check for defunct process before waiting for it 2018-07-23 20:58:25 +02:00
Cyrill Troxler
82ab4b0983 Add experimental s3backer mounter 
This also adds some generic handling of stale umounts.
Fuse returns immediately and does not indicate that
the mounter has finished writing to the backend.
The process finding is sort of hacky as I could not
find a better way to get to the PID from a fuse mount.
 2018-07-23 20:58:25 +02:00
Cyrill Troxler
b412e81977 Fix s3ql unmount 2018-07-22 12:41:37 +02:00
Cyrill Troxler
8cd8f6b6cd Mounter can now only be specified through flag 2018-07-22 12:41:37 +02:00
Cyrill Troxler
9d5d84ebfb Refactor all mounters to use the mounter interface 2018-07-22 12:41:37 +02:00
Cyrill Troxler
093c5bf500 Use ginkgo test suite to test all mounters 2018-07-22 12:41:37 +02:00
Cyrill Troxler
13eba47da6 Add experimental s3ql mounter 2018-07-22 12:39:54 +02:00