@mcpher/gas-fakes 1.1.5 → 1.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -89
- package/env.setup.template +16 -0
- package/package.json +2 -2
- package/src/services/advdocs/docapis.js +1 -1
- package/src/services/advdrive/drapis.js +1 -1
- package/src/services/advforms/formsapis.js +1 -1
- package/src/services/advgmail/fakeadvgmaildrafts.js +31 -0
- package/src/services/advgmail/fakeadvgmailusers.js +5 -0
- package/src/services/advgmail/gmailapis.js +1 -1
- package/src/services/advsheets/shapis.js +1 -1
- package/src/services/advslides/slapis.js +1 -1
- package/src/services/gmailapp/fakegmailapp.js +26 -0
- package/src/services/gmailapp/fakegmaildraft.js +30 -0
- package/src/support/auth.js +20 -25
- package/src/support/sxauth.js +7 -32
- package/src/support/sxdrive.js +2 -0
- package/src/support/syncit.js +0 -2
- package/src/support/workersync/synchronizer.js +52 -16
- package/src/support/workersync/worker.js +23 -14
package/README.md
CHANGED
|
@@ -1,110 +1,37 @@
|
|
|
1
|
-
# A proof of concept implementation of Apps Script Environment on Node
|
|
1
|
+
# <img src="./logo.png" alt="gas-fakes logo" width="50" align="top"> A proof of concept implementation of Apps Script Environment on Node
|
|
2
2
|
|
|
3
|
-
I use clasp/vscode to develop Google Apps Script (GAS) applications, but when using GAS native services, there's way too much back and
|
|
3
|
+
I use clasp/vscode to develop Google Apps Script (GAS) applications, but when using GAS native services, there's way too much back and forwards to the GAS IDE going while testing. I set myself the ambition of implementing a fake version of the GAS runtime environment on Node so I could at least do some testing and debugging of Apps Scripts locally on Node.
|
|
4
4
|
|
|
5
5
|
This is a proof of concept so I've implemented a growing subset of number of services and methods. There are a rigorous set of tests for all emulated classes and methods to make sure the same code produces the same result on both Node and Apps Script. Please report any inconsistencies in the issues of this repo.
|
|
6
6
|
|
|
7
7
|
|
|
8
8
|
## Getting started as a package user
|
|
9
9
|
|
|
10
|
-
You can get the package from npm
|
|
11
|
-
|
|
10
|
+
You can get the package from npm:
|
|
12
11
|
```sh
|
|
13
12
|
npm i @mcpher/gas-fakes
|
|
14
13
|
```
|
|
15
14
|
|
|
16
|
-
|
|
15
|
+
For a complete guide on how to set up your local environment for authentication and development, please see the consolidated guide: [Getting Started with `gas-fakes`](gas-fakes/GETTING_STARTED.md)
|
|
16
|
+
|
|
17
|
+
Collaborators should fork the repo and use the local versions of these files - see collaborators info.
|
|
17
18
|
|
|
18
19
|
### Use exactly the same code as in Apps Script
|
|
19
20
|
|
|
20
21
|
Just as on Apps Script, everything is executed synchronously so you don't need to bother with handling Promises/async/await. Just write normal Apps Script code. Usually you would have an associated App Script project if that's your eventual target, but it's not essential that you do. You can get started right away on Node.
|
|
21
22
|
|
|
22
|
-
|
|
23
|
-
### Cloud project
|
|
24
|
-
|
|
25
|
-
You don't have access to GAS maintained cloud projects from Node, so you'll need to create a GCP project to use locally (or you can use it on Apps Script too if you prefer) that has the workspace APIs enabled (Drive, Docs, Sheets etc).
|
|
26
|
-
|
|
27
|
-
### .env and shell script helpers
|
|
28
|
-
|
|
29
|
-
In order to duplicate the OAuth management handled by GAS, we'll use Application Default Credentials. I've provided a handy shell that will take care of all this for you.
|
|
30
|
-
|
|
31
|
-
- Get this [folder](https://github.com/brucemcpherson/gas-fakes/tree/main/shells) into the ./shells folder of your project.
|
|
32
|
-
- Get this [env template](https://github.com/brucemcpherson/gas-fakes/blob/main/shells/.env.template) and copy it/add it to your .env file in your project
|
|
33
|
-
|
|
34
|
-
#### Application default credentials
|
|
35
|
-
|
|
36
|
-
In order to avoid a bunch of Node specific code and credentials, yet still handle OAuth, I figured that we could simply rely on ADC. This is a problem I already wrote about here [Application Default Credentials with Google Cloud and Workspace APIs](https://ramblings.mcpher.com/application-default-credentials-with-google-cloud-and-workspace-apis/)
|
|
37
|
-
|
|
38
|
-
At the very least you need to add the gcp project id and optionally the id of some file you have access to - this'll be used to check that you have set up ADC properly.
|
|
39
|
-
|
|
40
|
-
#### Your .env file
|
|
41
|
-
|
|
42
|
-
You can setup the .env file your self using the .env.template as a guide.
|
|
43
|
-
|
|
44
|
-
```
|
|
45
|
-
# must set these
|
|
46
|
-
GCP_PROJECT_ID="add your gcp project id here"
|
|
47
|
-
|
|
48
|
-
# optional reference if you want to run a test after setting up
|
|
49
|
-
DRIVE_TEST_FILE_ID="add the id of some test file you have access to here"
|
|
50
|
-
|
|
51
|
-
# we'll use the default config for application default credentials
|
|
52
|
-
# probably dont need to change these
|
|
53
|
-
AC=default
|
|
54
|
-
# these are the scopes set by default - take some of these out if you want to minimize access
|
|
55
|
-
DEFAULT_SCOPES="https://www.googleapis.com/auth/userinfo.email,openid,https://www.googleapis.com/auth/cloud-platform"
|
|
56
|
-
EXTRA_SCOPES=",https://www.googleapis.com/auth/drive,https://www.googleapis.com/auth/spreadsheets"
|
|
57
|
-
|
|
58
|
-
# optional logging destination
|
|
59
|
-
# can be CONSOLE (default), CLOUD, BOTH, NONE
|
|
60
|
-
LOG_DESTINATION="BOTH"
|
|
61
|
-
|
|
62
|
-
```
|
|
63
|
-
#### Applying the .env
|
|
64
|
-
|
|
65
|
-
You can run
|
|
66
|
-
```bash
|
|
67
|
-
cd shells
|
|
68
|
-
bash setaccounts.sh
|
|
69
|
-
```
|
|
70
|
-
Which will set up and test the initial application default login with the selected scopes.
|
|
71
|
-
|
|
72
|
-
#### enabling workspace services
|
|
73
|
-
|
|
74
|
-
You can run this shell to enable all the required cloud services
|
|
75
|
-
```bash
|
|
76
|
-
cd shells
|
|
77
|
-
bash enable.sh
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
#### Restricted scopes
|
|
81
|
-
|
|
82
|
-
Recent changes in the OAuth security model have made it more difficult for Application Default credentials to work with all the scopes we'll need to fully emulate Live Apps Script locally. However there is a way to work around this by creating an oauth client for internal use in the cloud console, and injecting its credentials into those used by the application default creadentials process. You should now follow the guidance in [restricted scopes](restricted_scopes.md) to enhance your login process to be able access all the supported services in gas-fakes
|
|
83
|
-
|
|
84
|
-
#### Manifest file
|
|
85
|
-
|
|
86
|
-
If you have an associated apps script project, you'll probably be using clasp to sync with the apps script IDE, and you'll have an appsscript.json available in your project folder
|
|
87
|
-
|
|
88
|
-
**gas-fakes** reads the manifest file to see which scopes you need in your project, uses the Google Auth library to attempt to authorizes them and has `ScriptApp.getOauthToken()` return a sufficiently specced token, just as the GAS environment does. Just make sure you have an `appsscript.json` in the same folder as your main script.
|
|
89
|
-
|
|
90
|
-
Now you can execute this and it will set up your ADC to be able to run any services that require the scopes you add.
|
|
91
|
-
|
|
92
|
-
##### note
|
|
93
|
-
|
|
94
|
-
Although you may be tempted to add `https://www.googleapis.com/auth/script.external_request`, it's not necessary for the ADC and in fact will generate an error. You will of course need it in your Apps script manifest. Same goes for "https://www.googleapis.com/auth/documents" and "https://www.googleapis.com/auth/documents"
|
|
95
|
-
|
|
96
23
|
### Settings
|
|
97
24
|
|
|
98
|
-
|
|
25
|
+
The optional `gasfakes.json` file holds various location and behavior parameters for your local Node environment. It is not required on GAS, as you can't change anything over there. If you don't provide this file, `gas-fakes` will create one for you with sensible defaults.
|
|
99
26
|
|
|
100
|
-
```
|
|
27
|
+
```json
|
|
101
28
|
{
|
|
102
29
|
"manifest": "./appsscript.json",
|
|
103
30
|
"clasp": "./.clasp.json",
|
|
104
31
|
"documentId": null,
|
|
105
32
|
"cache": "/tmp/gas-fakes/cache",
|
|
106
33
|
"properties": "/tmp/gas-fakes/properties",
|
|
107
|
-
"scriptId": "
|
|
34
|
+
"scriptId": "a-unique-id-for-your-local-project"
|
|
108
35
|
}
|
|
109
36
|
```
|
|
110
37
|
|
|
@@ -182,7 +109,7 @@ If you want to set an initial LOG_DESTINATION using that .env file, you have to
|
|
|
182
109
|
```env
|
|
183
110
|
node --env-file=.env yourapp.js
|
|
184
111
|
```
|
|
185
|
-
Some developers prefer to use
|
|
112
|
+
Some developers prefer to use dotenv to set the path of the .env file
|
|
186
113
|
```javascript
|
|
187
114
|
import dotenv from 'dotenv'
|
|
188
115
|
dotenv.config({ path: '/custom/path/to/.env' })
|
|
@@ -213,11 +140,12 @@ For inspiration on pushing modified files to the IDE, see the togas.sh bash scri
|
|
|
213
140
|
|
|
214
141
|
As I mentioned earlier, to take this further, I'm going to need a lot of help to extend the methods and services supported - so if you feel this would be useful to you, and would like to collaborate, please ping me on bruce@mcpher.com and we'll talk.
|
|
215
142
|
|
|
216
|
-
##
|
|
217
|
-
|
|
143
|
+
## <img src="./logo.png" alt="gas-fakes logo" width="50" align="top"> Further Reading
|
|
218
144
|
|
|
145
|
+
- [getting started](GETTING_STARTED.md) - how to handle authentication for restricted scopes.
|
|
146
|
+
- [readme](README.md)
|
|
219
147
|
- [initial idea and thoughts](https://ramblings.mcpher.com/a-proof-of-concept-implementation-of-apps-script-environment-on-node/)
|
|
220
|
-
- [Inside the volatile world of a Google Document](https://ramblings.mcpher.com/inside-the-volatile-world-of-a-google-document/
|
|
148
|
+
- [Inside the volatile world of a Google Document](https://ramblings.mcpher.com/inside-the-volatile-world-of-a-google-document/)
|
|
221
149
|
- [Apps Script Services on Node – using apps script libraries](https://ramblings.mcpher.com/apps-script-services-on-node-using-apps-script-libraries/)
|
|
222
150
|
- [Apps Script environment on Node – more services](https://ramblings.mcpher.com/apps-script-environment-on-node-more-services/)
|
|
223
151
|
- [Turning async into synch on Node using workers](https://ramblings.mcpher.com/turning-async-into-synch-on-node-using-workers/)
|
|
@@ -226,8 +154,7 @@ As I mentioned earlier, to take this further, I'm going to need a lot of help to
|
|
|
226
154
|
- [colaborators](collaborators.md) - additional information for collaborators
|
|
227
155
|
- [oddities](oddities.md) - a collection of oddities uncovered during this project
|
|
228
156
|
- [gemini](gemini.md) - some reflections and experiences on using gemini to help code large projects
|
|
229
|
-
- [named colors](named-colors.md) - colors supported by Apps Script
|
|
230
|
-
- [this file](README.md)
|
|
231
157
|
- [named colors](named-colors.md)
|
|
232
158
|
- [sandbox](sandbox.md)
|
|
233
|
-
- [named range identity](named-range-identity.md)
|
|
159
|
+
- [named range identity](named-range-identity.md)
|
|
160
|
+
- [adc and restricted scopes](https://ramblings.mcpher.com/how-to-allow-access-to-sensitive-scopes-with-application-default-credentials/)
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# for testing authentication
|
|
2
|
+
GCP_PROJECT_ID="add your gcp project id here"
|
|
3
|
+
|
|
4
|
+
# this is optional
|
|
5
|
+
DRIVE_TEST_FILE_ID="add the id of some test file you have access to here"
|
|
6
|
+
|
|
7
|
+
# we'll use the default config for application default credentials
|
|
8
|
+
AC=default
|
|
9
|
+
# these are the scopes set by default - take some of these out if you want to minimize access
|
|
10
|
+
DEFAULT_SCOPES="https://www.googleapis.com/auth/userinfo.email,openid,https://www.googleapis.com/auth/cloud-platform"
|
|
11
|
+
EXTRA_SCOPES=",https://www.googleapis.com/auth/drive,https://www.googleapis.com/auth/spreadsheets,https://www.googleapis.com/auth/gmail.labels"
|
|
12
|
+
|
|
13
|
+
# if you need access to sensitive or restricted scopes you'll need to provide a credentials file
|
|
14
|
+
# this is a path relative to the project root, or an absolute path
|
|
15
|
+
CLIENT_CREDENTIAL_FILE="private/gasfakes-oa.json"
|
|
16
|
+
LOG_DESTINATION="CONSOLE"
|
package/package.json
CHANGED
|
@@ -59,13 +59,13 @@
|
|
|
59
59
|
"testdocsfootnotes": "node ./test/testdocsfootnotes.js execute",
|
|
60
60
|
"testdocsimages": "node ./test/testdocsimages.js execute",
|
|
61
61
|
"testdocsstyles": "node ./test/testdocsstyles.js execute",
|
|
62
|
-
"testsandbox": "node ./test/testsandbox.js execute",
|
|
62
|
+
"testsandbox": "node --trace-warnings ./test/testsandbox.js execute",
|
|
63
63
|
"testgmail": "node ./test/testgmail.js execute",
|
|
64
64
|
"testlogger": "node ./test/testlogger.js execute",
|
|
65
65
|
"pub": "npm publish --access public"
|
|
66
66
|
},
|
|
67
67
|
"name": "@mcpher/gas-fakes",
|
|
68
|
-
"version": "1.1.
|
|
68
|
+
"version": "1.1.6",
|
|
69
69
|
"license": "MIT",
|
|
70
70
|
"main": "main.js",
|
|
71
71
|
"description": "A proof of concept implementation of Apps Script Environment on Node",
|
|
@@ -5,7 +5,7 @@ import { syncLog} from '../../support/workersync/synclogger.js'
|
|
|
5
5
|
let __client = null;
|
|
6
6
|
|
|
7
7
|
export const getDocsApiClient = () => {
|
|
8
|
-
const auth = Auth.
|
|
8
|
+
const auth = Auth.getAuthClient()
|
|
9
9
|
if (!__client) {
|
|
10
10
|
syncLog('Creating new Docs API client');
|
|
11
11
|
__client = google.docs({ version: 'v1', auth });
|
|
@@ -5,7 +5,7 @@ import { syncLog} from '../../support/workersync/synclogger.js'
|
|
|
5
5
|
let __client = null;
|
|
6
6
|
syncLog('...importing Drive API');
|
|
7
7
|
export const getDriveApiClient = () => {
|
|
8
|
-
const auth = Auth.
|
|
8
|
+
const auth = Auth.getAuthClient()
|
|
9
9
|
if (!__client) {
|
|
10
10
|
syncLog('Creating new Drive API client');
|
|
11
11
|
__client = google.drive({ version: 'v3', auth });
|
|
@@ -5,7 +5,7 @@ import { syncLog} from '../../support/workersync/synclogger.js'
|
|
|
5
5
|
let __client = null;
|
|
6
6
|
|
|
7
7
|
export const getFormsApiClient = () => {
|
|
8
|
-
const auth = Auth.
|
|
8
|
+
const auth = Auth.getAuthClient()
|
|
9
9
|
if (!__client) {
|
|
10
10
|
syncLog('Creating new Forms API client');
|
|
11
11
|
__client = google.forms({ version: 'v1', auth });
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
import { FakeAdvResource } from '../common/fakeadvresource.js';
|
|
2
|
+
import { Proxies } from '../../support/proxies.js';
|
|
3
|
+
import { gError, normalizeSerialization } from '../../support/helpers.js';
|
|
4
|
+
import { Syncit } from '../../support/syncit.js';
|
|
5
|
+
|
|
6
|
+
export const newFakeAdvGmailDrafts = (...args) => Proxies.guard(new FakeAdvGmailDrafts(...args));
|
|
7
|
+
|
|
8
|
+
class FakeAdvGmailDrafts extends FakeAdvResource {
|
|
9
|
+
constructor(mainService) {
|
|
10
|
+
super(mainService, 'users', Syncit.fxGmail);
|
|
11
|
+
this.gmail = mainService;
|
|
12
|
+
this.__fakeObjectType = 'Gmail.Users.Drafts';
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
/**
|
|
16
|
+
* Creates a new draft with the DRAFT label.
|
|
17
|
+
* @param {object} resource - The draft resource to create.
|
|
18
|
+
* @param {string} userId - The user's email address. The special value me can be used to indicate the authenticated user.
|
|
19
|
+
* @returns {object} The created draft resource.
|
|
20
|
+
*/
|
|
21
|
+
create(resource, userId) {
|
|
22
|
+
const { data, response } = this._call(
|
|
23
|
+
'create',
|
|
24
|
+
{ userId, requestBody: normalizeSerialization(resource) },
|
|
25
|
+
null,
|
|
26
|
+
'drafts'
|
|
27
|
+
);
|
|
28
|
+
gError(response, 'gmail', 'users.drafts.create');
|
|
29
|
+
return data;
|
|
30
|
+
}
|
|
31
|
+
}
|
|
@@ -2,6 +2,7 @@ import { FakeAdvResource } from '../common/fakeadvresource.js';
|
|
|
2
2
|
import { Syncit } from '../../support/syncit.js';
|
|
3
3
|
import { signatureArgs, ssError, gError } from '../../support/helpers.js';
|
|
4
4
|
import { newFakeAdvGmailLabels } from './fakeadvgmaillabels.js';
|
|
5
|
+
import { newFakeAdvGmailDrafts } from './fakeadvgmaildrafts.js';
|
|
5
6
|
|
|
6
7
|
import { Proxies } from '../../support/proxies.js';
|
|
7
8
|
import { Utils } from '../../support/utils.js';
|
|
@@ -20,4 +21,8 @@ class FakeAdvGmailUsers extends FakeAdvResource {
|
|
|
20
21
|
return newFakeAdvGmailLabels(this.gmail);
|
|
21
22
|
}
|
|
22
23
|
|
|
24
|
+
get Drafts() {
|
|
25
|
+
return newFakeAdvGmailDrafts(this.gmail);
|
|
26
|
+
}
|
|
27
|
+
|
|
23
28
|
}
|
|
@@ -5,7 +5,7 @@ import { syncLog} from '../../support/workersync/synclogger.js'
|
|
|
5
5
|
let __client = null;
|
|
6
6
|
|
|
7
7
|
export const getGmailApiClient = () => {
|
|
8
|
-
const auth = Auth.
|
|
8
|
+
const auth = Auth.getAuthClient()
|
|
9
9
|
if (!__client) {
|
|
10
10
|
syncLog('Creating new Gmail API client');
|
|
11
11
|
__client = google.gmail({ version: 'v1', auth });
|
|
@@ -5,7 +5,7 @@ let currentAuth = null;
|
|
|
5
5
|
let __client = null;
|
|
6
6
|
syncLog('...importing Sheets API');
|
|
7
7
|
export const getSheetsApiClient = () => {
|
|
8
|
-
const auth = Auth.
|
|
8
|
+
const auth = Auth.getAuthClient()
|
|
9
9
|
|
|
10
10
|
if (__client && currentAuth !== auth) {
|
|
11
11
|
syncLog('Auth has changed - creating new Sheets API client');
|
|
@@ -6,7 +6,7 @@ import { syncLog} from '../../support/workersync/synclogger.js'
|
|
|
6
6
|
let __client = null;
|
|
7
7
|
syncLog('...importing Slides API');
|
|
8
8
|
export const getSlidesApiClient = () => {
|
|
9
|
-
const auth = Auth.
|
|
9
|
+
const auth = Auth.getAuthClient()
|
|
10
10
|
if (!__client) {
|
|
11
11
|
syncLog('Creating new Slides API client');
|
|
12
12
|
__client = google.slides({ version: 'v1', auth });
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { Proxies } from '../../support/proxies.js';
|
|
2
2
|
import { newFakeGmailLabel } from './fakegmaillabel.js';
|
|
3
|
+
import { newFakeGmailDraft } from './fakegmaildraft.js';
|
|
3
4
|
|
|
4
5
|
/**
|
|
5
6
|
* Provides access to Gmail threads, messages, and labels.
|
|
@@ -9,6 +10,31 @@ class FakeGmailApp {
|
|
|
9
10
|
this.__fakeObjectType = 'GmailApp';
|
|
10
11
|
}
|
|
11
12
|
|
|
13
|
+
/**
|
|
14
|
+
* Creates a draft email message.
|
|
15
|
+
* @param {string} recipient a comma-separated list of email addresses
|
|
16
|
+
* @param {string} subject the subject of the message
|
|
17
|
+
* @param {string} body the body of the message
|
|
18
|
+
* @returns {GmailDraft} the newly created draft
|
|
19
|
+
*/
|
|
20
|
+
createDraft(recipient, subject, body) {
|
|
21
|
+
// this is a fairly naive implementation of rfc2822
|
|
22
|
+
const raw = [
|
|
23
|
+
`To: ${recipient}`,
|
|
24
|
+
`Subject: ${subject}`,
|
|
25
|
+
'Content-Type: text/plain; charset=utf-8',
|
|
26
|
+
'',
|
|
27
|
+
body,
|
|
28
|
+
].join('\r\n');
|
|
29
|
+
|
|
30
|
+
// rfc4648 url safe alphabet
|
|
31
|
+
const encoded = Utilities.base64Encode(raw, Utilities.Charset.UTF_8)
|
|
32
|
+
.replace(/\+/g, '-').replace(/\//g, '_');
|
|
33
|
+
|
|
34
|
+
const draft = Gmail.Users.Drafts.create({ message: { raw: encoded } }, 'me');
|
|
35
|
+
return newFakeGmailDraft(draft);
|
|
36
|
+
}
|
|
37
|
+
|
|
12
38
|
/**
|
|
13
39
|
* Creates a new user label.
|
|
14
40
|
* @param {string} name The name of the new label.
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
import { Proxies } from '../../support/proxies.js';
|
|
2
|
+
|
|
3
|
+
export const newFakeGmailDraft = (...args) => Proxies.guard(new FakeGmailDraft(...args));
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* A draft email message in a user's Gmail account.
|
|
7
|
+
* @see https://developers.google.com/apps-script/reference/gmail/gmail-draft
|
|
8
|
+
*/
|
|
9
|
+
class FakeGmailDraft {
|
|
10
|
+
constructor(draftResource) {
|
|
11
|
+
this.__draftResource = draftResource;
|
|
12
|
+
this.__fakeObjectType = 'GmailDraft';
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
/**
|
|
16
|
+
* Gets the ID of this draft.
|
|
17
|
+
* @returns {string} The draft ID.
|
|
18
|
+
*/
|
|
19
|
+
getId() {
|
|
20
|
+
return this.__draftResource.id;
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
/**
|
|
24
|
+
* Returns "GmailDraft"
|
|
25
|
+
* @returns {string}
|
|
26
|
+
*/
|
|
27
|
+
toString() {
|
|
28
|
+
return this.__fakeObjectType;
|
|
29
|
+
}
|
|
30
|
+
}
|
package/src/support/auth.js
CHANGED
|
@@ -9,7 +9,6 @@ const _authScopes = new Set([])
|
|
|
9
9
|
let _auth = null
|
|
10
10
|
let _projectId = null
|
|
11
11
|
let _tokenInfo = null
|
|
12
|
-
let _adcPath = null
|
|
13
12
|
let _accessToken = null;
|
|
14
13
|
let _tokenExpiresAt = null;
|
|
15
14
|
let _manifest = null
|
|
@@ -24,8 +23,6 @@ const getSettings = () => _settings
|
|
|
24
23
|
const getScriptId = () => getSettings().scriptId
|
|
25
24
|
const getDocumentId = () => getSettings().documentId
|
|
26
25
|
const setProjectId = (projectId) => _projectId = projectId
|
|
27
|
-
const setAdcPath = (adcPath) => _adcPath = adcPath
|
|
28
|
-
const getAdcPath = () => _adcPath
|
|
29
26
|
const setAccessToken = (accessToken) => _accessToken = accessToken
|
|
30
27
|
const setSettings = (settings) => _settings = settings
|
|
31
28
|
const getCachePath = () => getSettings().cache
|
|
@@ -61,30 +58,29 @@ const isTokenExpired = () => !_accessToken || !_tokenExpiresAt || Date.now() >=
|
|
|
61
58
|
* @param {string[]} [scopes=[]] the required scopes will be added to existing scopes already asked for
|
|
62
59
|
* @returns {GoogleAuth.auth}
|
|
63
60
|
*/
|
|
64
|
-
|
|
61
|
+
let _authClient = null
|
|
62
|
+
const getAuthClient = () => _authClient
|
|
65
63
|
const setAuth = async (scopes = [], keyFile = null) => {
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
64
|
+
const hasCurrentAuth = hasAuth() && scopes.every(s => _authScopes.has(s));
|
|
65
|
+
|
|
66
|
+
|
|
67
|
+
if (!hasCurrentAuth) {
|
|
68
|
+
syncLog(`...initializing auth and discovering project ID`);
|
|
69
|
+
|
|
70
|
+
// 1. Create the GoogleAuth manager instance (this instance has getProjectId)
|
|
71
|
+
_auth = new GoogleAuth({ scopes });
|
|
72
|
+
|
|
73
|
+
// 2. Use the manager to get the authenticated client (this is passed to API methods)
|
|
74
|
+
_authClient = await _auth.getClient();
|
|
75
|
+
|
|
76
|
+
// 3. Use the manager to reliably get the project ID
|
|
77
|
+
_projectId = await _auth.getProjectId();
|
|
78
|
+
|
|
76
79
|
if (!_projectId) {
|
|
77
80
|
throw new Error('Failed to get project ID from Application Default Credentials.');
|
|
78
81
|
}
|
|
82
|
+
|
|
79
83
|
syncLog(`...discovered and set projectId to ${_projectId}`);
|
|
80
|
-
|
|
81
|
-
// Now, create the *real* auth client with the projectId and all required scopes.
|
|
82
|
-
_auth = new GoogleAuth({
|
|
83
|
-
keyFile,
|
|
84
|
-
scopes,
|
|
85
|
-
projectId: _projectId,
|
|
86
|
-
});
|
|
87
|
-
|
|
88
84
|
scopes.forEach(s => _authScopes.add(s));
|
|
89
85
|
}
|
|
90
86
|
return getAuth();
|
|
@@ -184,8 +180,6 @@ export const Auth = {
|
|
|
184
180
|
getAuthedScopes,
|
|
185
181
|
googify,
|
|
186
182
|
setProjectId,
|
|
187
|
-
setAdcPath,
|
|
188
|
-
getAdcPath,
|
|
189
183
|
getUserId,
|
|
190
184
|
setTokenInfo,
|
|
191
185
|
getAccessToken,
|
|
@@ -203,5 +197,6 @@ export const Auth = {
|
|
|
203
197
|
getClasp,
|
|
204
198
|
getTimeZone,
|
|
205
199
|
setAccessToken,
|
|
206
|
-
isTokenExpired
|
|
200
|
+
isTokenExpired,
|
|
201
|
+
getAuthClient
|
|
207
202
|
}
|
package/src/support/sxauth.js
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
*/
|
|
8
8
|
import got from 'got';
|
|
9
9
|
import { Auth } from './auth.js';
|
|
10
|
-
import { syncLog } from './workersync/synclogger.js';
|
|
10
|
+
import { syncError, syncLog } from './workersync/synclogger.js';
|
|
11
11
|
import { homedir } from 'os';
|
|
12
12
|
import {access, readFile, writeFile, mkdir } from 'fs/promises';
|
|
13
13
|
import path from 'path'
|
|
@@ -28,35 +28,8 @@ import path from 'path'
|
|
|
28
28
|
*/
|
|
29
29
|
export const sxInit = async ({ manifestPath, claspPath, settingsPath, mainDir, cachePath, propertiesPath, fakeId }) => {
|
|
30
30
|
|
|
31
|
-
const findAdcPath = async () => {
|
|
32
31
|
|
|
33
32
|
|
|
34
|
-
if (process.env.GOOGLE_APPLICATION_CREDENTIALS) {
|
|
35
|
-
return process.env.GOOGLE_APPLICATION_CREDENTIALS;
|
|
36
|
-
}
|
|
37
|
-
const isWindows = process.platform === 'win32';
|
|
38
|
-
const home = homedir();
|
|
39
|
-
let adcPath;
|
|
40
|
-
if (isWindows) {
|
|
41
|
-
const appData = process.env.APPDATA;
|
|
42
|
-
if (appData) {
|
|
43
|
-
adcPath = path.join(appData, 'gcloud', 'application_default_credentials.json');
|
|
44
|
-
}
|
|
45
|
-
} else {
|
|
46
|
-
adcPath = path.join(home, '.config', 'gcloud', 'application_default_credentials.json');
|
|
47
|
-
}
|
|
48
|
-
|
|
49
|
-
if (adcPath) {
|
|
50
|
-
try {
|
|
51
|
-
await access(adcPath);
|
|
52
|
-
return adcPath;
|
|
53
|
-
} catch {
|
|
54
|
-
// file doesn't exist or not accessible
|
|
55
|
-
}
|
|
56
|
-
}
|
|
57
|
-
return null;
|
|
58
|
-
};
|
|
59
|
-
|
|
60
33
|
// get the settings and manifest
|
|
61
34
|
|
|
62
35
|
|
|
@@ -118,17 +91,19 @@ export const sxInit = async ({ manifestPath, claspPath, settingsPath, mainDir, c
|
|
|
118
91
|
// Initialize auth. This is async and will discover the project ID.
|
|
119
92
|
const auth = await Auth.setAuth(scopes);
|
|
120
93
|
const projectId = Auth.getProjectId();
|
|
121
|
-
const adcPath = await findAdcPath();
|
|
122
94
|
const accessToken = await auth.getAccessToken()
|
|
123
|
-
|
|
124
|
-
|
|
95
|
+
let tokenInfo = null
|
|
96
|
+
try {
|
|
97
|
+
tokenInfo = await got(`https://www.googleapis.com/oauth2/v3/tokeninfo?access_token=${accessToken}`).json()
|
|
98
|
+
} catch (err) {
|
|
99
|
+
syncError (`failed to get access token info`)
|
|
100
|
+
}
|
|
125
101
|
|
|
126
102
|
/// these all jst exist in this sub process so we need to send them back to parent process
|
|
127
103
|
return {
|
|
128
104
|
scopes,
|
|
129
105
|
projectId,
|
|
130
106
|
tokenInfo,
|
|
131
|
-
adcPath,
|
|
132
107
|
accessToken, // also return the token itself
|
|
133
108
|
settings,
|
|
134
109
|
manifest,
|
package/src/support/sxdrive.js
CHANGED
|
@@ -11,6 +11,8 @@ import intoStream from 'into-stream';
|
|
|
11
11
|
import { getStreamAsBuffer } from 'get-stream';
|
|
12
12
|
import { syncWarn, syncError } from './workersync/synclogger.js';
|
|
13
13
|
import { getDriveApiClient } from '../services/advdrive/drapis.js';
|
|
14
|
+
|
|
15
|
+
|
|
14
16
|
/**
|
|
15
17
|
* serializable reponse from a sync call
|
|
16
18
|
* @typedef SxResponse
|
package/src/support/syncit.js
CHANGED
|
@@ -284,7 +284,6 @@ const fxInit = ({
|
|
|
284
284
|
scopes,
|
|
285
285
|
projectId,
|
|
286
286
|
tokenInfo,
|
|
287
|
-
adcPath,
|
|
288
287
|
accessToken,
|
|
289
288
|
settings,
|
|
290
289
|
manifest,
|
|
@@ -295,7 +294,6 @@ const fxInit = ({
|
|
|
295
294
|
Auth.setProjectId(projectId);
|
|
296
295
|
//Auth.setAuth(scopes)
|
|
297
296
|
Auth.setTokenInfo(tokenInfo);
|
|
298
|
-
Auth.setAdcPath(adcPath)
|
|
299
297
|
//Auth.setAccessToken(accessToken)
|
|
300
298
|
Auth.setSettings(settings);
|
|
301
299
|
Auth.setClasp(clasp);
|
|
@@ -6,12 +6,42 @@ import fs from 'node:fs';
|
|
|
6
6
|
const __filename = fileURLToPath(import.meta.url);
|
|
7
7
|
const __dirname = path.dirname(__filename);
|
|
8
8
|
|
|
9
|
+
// --- Start: Suppress google-auth-library warnings globally ---
|
|
10
|
+
// A regex to match either of the Google Auth deprecation warnings.
|
|
11
|
+
const googleAuthWarningRegex = /The `from(Stream|JSON)` method is deprecated/;
|
|
12
|
+
|
|
13
|
+
// Monkey-patch the main process's write methods to filter output.
|
|
14
|
+
const patchStream = (stream) => {
|
|
15
|
+
const originalWrite = stream.write;
|
|
16
|
+
stream.write = (chunk, encoding, callback) => {
|
|
17
|
+
const message = typeof chunk === 'string' ? chunk : chunk.toString();
|
|
18
|
+
if (googleAuthWarningRegex.test(message)) {
|
|
19
|
+
// If it's a warning we want to suppress, do nothing.
|
|
20
|
+
return true;
|
|
21
|
+
}
|
|
22
|
+
// Otherwise, call the original write method.
|
|
23
|
+
return originalWrite.apply(stream, [chunk, encoding, callback]);
|
|
24
|
+
};
|
|
25
|
+
};
|
|
26
|
+
|
|
27
|
+
patchStream(process.stdout);
|
|
28
|
+
patchStream(process.stderr);
|
|
29
|
+
// --- End: Suppress google-auth-library warnings ---
|
|
30
|
+
|
|
31
|
+
// Define indices for the control buffer to avoid magic numbers.
|
|
32
|
+
const CONTROL_INDICES = {
|
|
33
|
+
STATUS: 0, // 0: free, 1: busy, 2: worker_init
|
|
34
|
+
DATA_SIZE: 1, // Size of the result data in bytes
|
|
35
|
+
IS_ERROR: 2, // 0: success, 1: error
|
|
36
|
+
RESULT_TYPE: 3, // 0: buffer, 1: file
|
|
37
|
+
};
|
|
38
|
+
|
|
9
39
|
// Shared buffer for control signals
|
|
10
40
|
// [0]: status lock (0: free, 1: busy, 2: worker_init)
|
|
11
41
|
// [1]: result data size in bytes
|
|
12
42
|
// [2]: error flag (0: success, 1: error)
|
|
13
43
|
// [3]: result type (0: buffer, 1: file)
|
|
14
|
-
const controlBuf = new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT *
|
|
44
|
+
const controlBuf = new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT * Object.keys(CONTROL_INDICES).length);
|
|
15
45
|
const control = new Int32Array(controlBuf);
|
|
16
46
|
|
|
17
47
|
// Shared buffer for data transfer (e.g., 16MB)
|
|
@@ -24,7 +54,7 @@ const textDecoder = new TextDecoder();
|
|
|
24
54
|
// The single, long-lived worker
|
|
25
55
|
const worker = new Worker(path.resolve(__dirname, 'worker.js'));
|
|
26
56
|
|
|
27
|
-
// Pipe worker's
|
|
57
|
+
// Pipe worker's output directly. The patch above will handle filtering.
|
|
28
58
|
worker.stdout.pipe(process.stdout);
|
|
29
59
|
worker.stderr.pipe(process.stderr);
|
|
30
60
|
|
|
@@ -33,7 +63,7 @@ worker.stderr.pipe(process.stderr);
|
|
|
33
63
|
worker.on('error', (error) => {
|
|
34
64
|
console.error('Worker thread encountered a fatal error:', error);
|
|
35
65
|
// Unblock any pending Atomics.wait() call to prevent a deadlock.
|
|
36
|
-
Atomics.store(control,
|
|
66
|
+
Atomics.store(control, CONTROL_INDICES.STATUS, 0); // Set status to "free"
|
|
37
67
|
Atomics.notify(control, 0);
|
|
38
68
|
// Exit the main process, as the application is in an unrecoverable state.
|
|
39
69
|
process.exit(1);
|
|
@@ -44,24 +74,30 @@ worker.on('exit', (code) => {
|
|
|
44
74
|
// This handles cases where the worker exits unexpectedly.
|
|
45
75
|
if (code !== 0) {
|
|
46
76
|
console.error(`Worker thread stopped with exit code ${code}`);
|
|
47
|
-
Atomics.store(control,
|
|
77
|
+
Atomics.store(control, CONTROL_INDICES.STATUS, 0); // Unblock main thread
|
|
48
78
|
Atomics.notify(control, 0);
|
|
49
79
|
}
|
|
50
80
|
});
|
|
51
81
|
|
|
52
82
|
// Send the shared buffers to the worker once and wait for it to confirm initialization
|
|
53
|
-
Atomics.store(control,
|
|
83
|
+
Atomics.store(control, CONTROL_INDICES.STATUS, 2); // Set status to "worker_init"
|
|
54
84
|
worker.postMessage({ controlBuf, dataBuf });
|
|
55
|
-
Atomics.wait(control,
|
|
85
|
+
Atomics.wait(control, CONTROL_INDICES.STATUS, 2); // Wait for worker to set status to "free" (0)
|
|
56
86
|
|
|
57
87
|
// Allow the main process to exit even if the worker is still running.
|
|
58
88
|
worker.unref();
|
|
59
89
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
90
|
+
/**
|
|
91
|
+
* Ensures the worker is terminated when the main process exits,
|
|
92
|
+
* whether normally or via signals like Ctrl+C.
|
|
93
|
+
*/
|
|
94
|
+
function cleanup() {
|
|
95
|
+
console.log('...terminating worker thread');
|
|
96
|
+
worker.terminate();
|
|
97
|
+
}
|
|
98
|
+
process.on('exit', cleanup);
|
|
99
|
+
process.on('SIGINT', cleanup); // Catches Ctrl+C
|
|
100
|
+
process.on('SIGTERM', cleanup); // Catches `kill`
|
|
65
101
|
|
|
66
102
|
/**
|
|
67
103
|
* Calls an async function in the worker and blocks until it returns.
|
|
@@ -74,7 +110,7 @@ export function callSync(method, ...args) {
|
|
|
74
110
|
|
|
75
111
|
|
|
76
112
|
// 1. Set status to "busy". This acts as a lock.
|
|
77
|
-
Atomics.store(control,
|
|
113
|
+
Atomics.store(control, CONTROL_INDICES.STATUS, 1);
|
|
78
114
|
|
|
79
115
|
// 2. Send the task to the worker.
|
|
80
116
|
const payload = { method, args };
|
|
@@ -83,12 +119,12 @@ export function callSync(method, ...args) {
|
|
|
83
119
|
// 3. Block and wait for the worker to finish.
|
|
84
120
|
// It's "busy" (1) until the worker sets it back to "free" (0).
|
|
85
121
|
// This is a true blocking wait, consuming minimal CPU.
|
|
86
|
-
Atomics.wait(control,
|
|
122
|
+
Atomics.wait(control, CONTROL_INDICES.STATUS, 1);
|
|
87
123
|
|
|
88
124
|
// 4. Worker is done, result is in the shared buffer.
|
|
89
|
-
const resultSize = Atomics.load(control,
|
|
90
|
-
const hasError = Atomics.load(control,
|
|
91
|
-
const resultIsFile = Atomics.load(control,
|
|
125
|
+
const resultSize = Atomics.load(control, CONTROL_INDICES.DATA_SIZE);
|
|
126
|
+
const hasError = Atomics.load(control, CONTROL_INDICES.IS_ERROR) === 1;
|
|
127
|
+
const resultIsFile = Atomics.load(control, CONTROL_INDICES.RESULT_TYPE) === 1;
|
|
92
128
|
|
|
93
129
|
if (resultSize > dataBuf.byteLength) {
|
|
94
130
|
throw new Error(
|
|
@@ -10,6 +10,14 @@ let control;
|
|
|
10
10
|
let dataView;
|
|
11
11
|
const textEncoder = new TextEncoder();
|
|
12
12
|
|
|
13
|
+
// Define indices for the control buffer to match synchronizer.js
|
|
14
|
+
const CONTROL_INDICES = {
|
|
15
|
+
STATUS: 0, // 0: free, 1: busy, 2: worker_init
|
|
16
|
+
DATA_SIZE: 1, // Size of the result data in bytes
|
|
17
|
+
IS_ERROR: 2, // 0: success, 1: error
|
|
18
|
+
RESULT_TYPE: 3, // 0: buffer, 1: file
|
|
19
|
+
};
|
|
20
|
+
|
|
13
21
|
/**
|
|
14
22
|
* Writes a result to the shared buffer and sets control flags.
|
|
15
23
|
* @param {*} result The successful result to write.
|
|
@@ -27,16 +35,16 @@ async function writeResult(result) {
|
|
|
27
35
|
|
|
28
36
|
// Write the path to the shared buffer
|
|
29
37
|
dataView.set(pathBytes);
|
|
30
|
-
Atomics.store(control,
|
|
31
|
-
Atomics.store(control,
|
|
32
|
-
Atomics.store(control,
|
|
38
|
+
Atomics.store(control, CONTROL_INDICES.DATA_SIZE, pathBytes.length); // data size (of the path)
|
|
39
|
+
Atomics.store(control, CONTROL_INDICES.IS_ERROR, 0); // success flag
|
|
40
|
+
Atomics.store(control, CONTROL_INDICES.RESULT_TYPE, 1); // result type: file
|
|
33
41
|
} else {
|
|
34
42
|
// Result fits in the buffer, write it directly.
|
|
35
43
|
|
|
36
44
|
dataView.set(encodedResult);
|
|
37
|
-
Atomics.store(control,
|
|
38
|
-
Atomics.store(control,
|
|
39
|
-
Atomics.store(control,
|
|
45
|
+
Atomics.store(control, CONTROL_INDICES.DATA_SIZE, encodedResult.length); // data size
|
|
46
|
+
Atomics.store(control, CONTROL_INDICES.IS_ERROR, 0); // success flag
|
|
47
|
+
Atomics.store(control, CONTROL_INDICES.RESULT_TYPE, 0); // result type: buffer
|
|
40
48
|
|
|
41
49
|
}
|
|
42
50
|
}
|
|
@@ -55,9 +63,9 @@ function writeError(error) {
|
|
|
55
63
|
const encodedError = textEncoder.encode(errorString);
|
|
56
64
|
|
|
57
65
|
dataView.set(encodedError);
|
|
58
|
-
Atomics.store(control,
|
|
59
|
-
Atomics.store(control,
|
|
60
|
-
Atomics.store(control,
|
|
66
|
+
Atomics.store(control, CONTROL_INDICES.DATA_SIZE, encodedError.length); // data size
|
|
67
|
+
Atomics.store(control, CONTROL_INDICES.IS_ERROR, 1); // error flag
|
|
68
|
+
Atomics.store(control, CONTROL_INDICES.RESULT_TYPE, 0); // result type: buffer (errors are always small enough)
|
|
61
69
|
}
|
|
62
70
|
|
|
63
71
|
// 1. Receive the shared buffers from the main thread on startup.
|
|
@@ -66,7 +74,7 @@ parentPort.once('message', (msg) => {
|
|
|
66
74
|
dataView = new Uint8Array(msg.dataBuf);
|
|
67
75
|
|
|
68
76
|
// Signal that the worker is ready.
|
|
69
|
-
Atomics.store(control,
|
|
77
|
+
Atomics.store(control, CONTROL_INDICES.STATUS, 0);
|
|
70
78
|
Atomics.notify(control, 0);
|
|
71
79
|
});
|
|
72
80
|
|
|
@@ -79,7 +87,9 @@ const handleUncaughtError = (error) => {
|
|
|
79
87
|
// If control is not initialized, we can't report the error.
|
|
80
88
|
// Just log it and exit.
|
|
81
89
|
if (control) {
|
|
82
|
-
|
|
90
|
+
// Log the full error stack to identify the call site
|
|
91
|
+
syncError('A fatal, unhandled error occurred in the worker. The worker will be unresponsive.', error);
|
|
92
|
+
|
|
83
93
|
writeError(error);
|
|
84
94
|
Atomics.notify(control, 0);
|
|
85
95
|
}
|
|
@@ -105,8 +115,7 @@ parentPort.on('message', async (task) => {
|
|
|
105
115
|
// sxInit is special: it creates the auth state and returns serializable info.
|
|
106
116
|
result = await asyncFn(...task.args);
|
|
107
117
|
|
|
108
|
-
|
|
109
|
-
Auth.setAdcPath(result.adcPath);
|
|
118
|
+
|
|
110
119
|
// The projectId is already discovered and set within the initial `sxInit` -> `setAuth` call.
|
|
111
120
|
// This subsequent call is redundant, so it is removed for clarity.
|
|
112
121
|
// Auth.setProjectId(result.projectId);
|
|
@@ -132,7 +141,7 @@ parentPort.on('message', async (task) => {
|
|
|
132
141
|
writeError(error);
|
|
133
142
|
} finally {
|
|
134
143
|
// 3. Signal completion and wake up the main thread.
|
|
135
|
-
Atomics.store(control,
|
|
144
|
+
Atomics.store(control, CONTROL_INDICES.STATUS, 0);
|
|
136
145
|
Atomics.notify(control, 0);
|
|
137
146
|
}
|
|
138
147
|
});
|