screwdriver-api 7.0.212 → 7.0.214
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -863,7 +863,7 @@ We will introduce new commands to validate, publish and tag pipeline templates b
|
|
|
863
863
|
|
|
864
864
|
CLI can be installed using below command:
|
|
865
865
|
|
|
866
|
-
``` $ npm install
|
|
866
|
+
``` $ npm install screwdriver-template-main ```
|
|
867
867
|
|
|
868
868
|
|
|
869
869
|
***1. Validate***
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# SD Download Zipped Artifacts
|
|
2
|
+
|
|
3
|
+
#### Context
|
|
4
|
+
|
|
5
|
+
The current [API endpoint](https://github.com/screwdriver-cd/screwdriver/blob/master/plugins/builds/artifacts/get.js) allows downloading artifacts for a single file. This is quite limited, as users often need to download multiple files simultaneously.
|
|
6
|
+
|
|
7
|
+
#### Proposal
|
|
8
|
+
|
|
9
|
+
###### Asynchronous Implementation (preferred for a foolproof solution)
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
sequenceDiagram
|
|
13
|
+
Client->>API: Download dir
|
|
14
|
+
API->>DB: Save request
|
|
15
|
+
API->>Client: Return 201 with a link to get download status
|
|
16
|
+
Zipper->>DB: Consume request
|
|
17
|
+
create participant Store
|
|
18
|
+
Zipper->>Store: Stream files
|
|
19
|
+
destroy Store
|
|
20
|
+
Zipper->>Store: Close connection
|
|
21
|
+
Zipper->>Zipper: zipping
|
|
22
|
+
create participant S3
|
|
23
|
+
Zipper->>S3: upload zipped file
|
|
24
|
+
destroy S3
|
|
25
|
+
S3->>Zipper: send the URL info
|
|
26
|
+
destroy Zipper
|
|
27
|
+
Zipper->>DB: update request with S3 URL
|
|
28
|
+
Client->>API: Download dir status
|
|
29
|
+
destroy DB
|
|
30
|
+
API->>DB: Get download URL based on request ID
|
|
31
|
+
API->>Client: Return 200 with S3 URL
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
###### Async Implementation Flows
|
|
35
|
+
|
|
36
|
+
1. User requests to download a directory.
|
|
37
|
+
2. The client sends a download directory request to the API.
|
|
38
|
+
3. The API saves the request in the database.
|
|
39
|
+
4. The API returns a 201 response to the client with a link to check the download status.
|
|
40
|
+
5. The Zipper service consumes the request from the database.
|
|
41
|
+
6. The Zipper service calls Store and streams the files from the store.
|
|
42
|
+
7. The Zipper service starts zipping the streamed files.
|
|
43
|
+
8. The Zipper service uploads the zipped file to S3.
|
|
44
|
+
9. The S3 sends the URL information back to the Zipper service.
|
|
45
|
+
10. The Zipper service updates the request in the database with the S3 URL.
|
|
46
|
+
11. The client sends a request to check the download directory status to the API.
|
|
47
|
+
12. The API retrieves the download URL based on the request ID from the database.
|
|
48
|
+
13. The API returns a 200 response to the client with the S3 URL.
|
|
49
|
+
|
|
50
|
+
###### Synchronous Implementation (fast development time)
|
|
51
|
+
|
|
52
|
+
```mermaid
|
|
53
|
+
sequenceDiagram
|
|
54
|
+
Client->>API: Download dir
|
|
55
|
+
loop Directory tree
|
|
56
|
+
API->>Store: Stream files
|
|
57
|
+
end
|
|
58
|
+
API->>API: zipping
|
|
59
|
+
API->>Client: return the zipped file
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
###### Synchronous Implementation Flows
|
|
63
|
+
|
|
64
|
+
1. User requests to download a directory.
|
|
65
|
+
2. The client sends a download directory request to the API.
|
|
66
|
+
3. The API begins streaming the files from the store in a loop, recursively traversing the entire directory tree.
|
|
67
|
+
4. The API zips the streamed files.
|
|
68
|
+
5. The API returns the zipped file to the client.
|
|
69
|
+
6. The client receives the zipped file.
|
|
70
|
+
|
|
71
|
+
#### Design Caveat
|
|
72
|
+
|
|
73
|
+
Depending on the number of files being streamed, there’s a potential risk of running into memory issues. To prevent this, we should implement a check to estimate the total download size, such as 1GB.
|