@mcpher/gas-fakes 1.0.9 → 1.0.10

package/README.md

@@ -2,7 +2,7 @@
 
 I use clasp/vscode to develop Google Apps Script (GAS) applications, but when using GAS native services, there's way too much back and fowards to the GAS IDE going while testing. I set myself the ambition of implementing fake version of the GAS runtime environment on Node so I could at least do some testing locally.
 
-This is just a proof of concept so I've just implemented a very limited number of services and methods, but the tricky parts are all in place so all that's left is a load of busy work (to which I heartily invite any interested collaborators).
+This is just a proof of concept so I've implemented a subset of number of services and methods, but the tricky parts are all in place so all that's left is a load of busy work (to which I heartily invite any interested collaborators).
 
 ## progress
 
@@ -32,7 +32,10 @@ I recommend you use the test project included in the repo to make sure all is se
 
 Note that I use a [unit tester](https://ramblings.mcpher.com/apps-script-test-runner-library-ported-to-node/) that runs in both GAS and Node, so the exact same tests will run in both environments. There are some example tests in the repo. Each test has been proved on both Node and GAS. There's also a shell (togas.sh) which will use clasp to push the test code to Apps Script.
 
-Each test can be run indivually (for example `npm run testdrive`) or all with `npm test`
+Each test can be run individually (for example `npm run testdrive`) or all with `npm test`
+
+Test settings and fixtures are in the .env file. Some readonly files are publicly shared and can be left with the example value in .env-template. Most files which are written are created and deleted afterwards on successful completion. They will be named something starting with --. 
+
 
 ### Settings
 
@@ -157,7 +160,7 @@ v1.0.8
 - `ScriptApp` - almost all
 - `UrlFetchApp` - 80%
 - `Utilities` - almost all
-- `Sheets` - 25%
+- `Sheets` - 50%
 - `SpreadsheetApp` - 60%
 - `CacheService` - 80%
 - `PropertiesService` - 80%
@@ -206,6 +209,12 @@ The documentId is only meaningful if you are working on a container bound scrip.
 
 As you will have noticed, there are various local support files for props/caching etc. Be careful that these do not get committed to a public repo if you are adding sensitive values to your stores. Note that the real user Id is not used when creating files, but rather an encrypted version of it. This avoids real user ids being revealed in your file system.
 
+## Debugging
+
+For conversion of async to sync, I'm spawing a subprocess using [make-synchronous](https://github.com/sindresorhus/make-synchronous). Out of the box it inherits the Node Options from the main process, so that means it'll try to run each subprocess in debug mode also. Bringing up and down the debugger each time takes forever, so I've temporaily modified my local version of make-synchronous to drop the debug inheritance. 
+
+If this makes it to the repo we can start to use it from there - see issue https://github.com/sindresorhus/make-synchronous/issues/14
+
 ## Noticed differences
 
 I'll make a note in thre repos issues on implementation differences. In the main will be slight differences in error message text, which I'll normalize over time, or where Apps Script has a fundamental obstacle. Please report any differences in behavior you find in the repo issues.
@@ -216,7 +225,20 @@ I've come across various Apps Script bugs/issues as I work through this which I'
 
 ## Oddities
 
-Just a few things I've come across when digging into the differences between what the sheets API and Apps Script do. Whether or not you use gas fakes, some of this stuff might be useful if you are using the Sheets API directly, or indeed the Sheets Advanced service.
+Just a few things I've come across when digging into the differences between what the sheets API and Apps Script do. Whether or not you use gas fakes, some of this stuff might be useful if you are using the Sheets API directly, or indeed the Sheets Advanced service. I'll just make a growing list of stuff I've found, in no particular order.
+
+
+### Note to collaborators
+If you are tempted to use Gemini as a shortcut to avoid reading the docs, I've found that it's pretty inaccurate and you can waste a huge amount of time taking what it says as gospel. You'll get used to seeing this apology from Gemini - 
+````
+You are absolutely correct! My sincerest apologies for that significant error.
+
+You've hit on a crucial detail that I completely missed, and I deeply appreciate you pointing it out and providing the correct documentation link.
+````
+
+And you eventually have to dig into the docs yourself to track down why something Gemini advised isn't working. 
+
+I'm just not bothering with at all now. It wastes more time than it saves.
 
 ### Formats and styles
 
@@ -229,7 +251,7 @@ This means that sometimes, for example, a font might be red in the UI, but Apps
 
 ### Values
 
-Just as with Formats, the actual value rendered might be different than the value stored. For example the number 1 might be displayed as '1' but returned as 1, and visa versa depending on the effective format for its range. I'm not entrely sure at this point the exact rules that getValues() applies, but this is what I've implemented - which appears to get the results most similar to App Script. I haven't figured out how to handles dates yet.
+Just as with Formats, the actual value rendered might be different than the value stored. For example the number 1 might be displayed as '1' but returned as 1, and visa versa depending on the effective format for its range. I'm not entrely sure at this point the exact rules that getValues() applies, but this is what I've implemented - which appears to get the results most similar to App Script. 
 
 Here is how I've implemented getting and setting values.
 
@@ -340,6 +362,114 @@ Since only relative versions of single dates are implemented in GAS, there's no
 
 CriteriaValues are stored as a string, exactly as typed by the user. This means that if the API is operating in a different locale to the sheet, date formats will be different and wrong (for example - 20/2/23 in UK is 2/20/23 in US). This is a problem you would anyway face in Apps Script so I don't plan to handle this right now.
 
+## Various hints when using the advanced sheets service
+
+I've tried to exactly imitate the behavior of the Sheets advanced service (even though it's often inconvenient and inconsistent), so these following comments apply to both Sheets and FakeSheets. If you are usng the Advanced service, here's a few hints Ive come across that might be helpful.
+
+### Advanced sheets updating cells
+
+The advanced sheets service provides a huge list of builders such as Sheets.newCellData(). This is supposed to simplify building requests using the Sheets service, rather than building the requests from scratch your self. I sometimes find them more long winded that just making the objects, and I notice that there are no checks on the values that you set using them, so there's not any validation to proft from. 
+
+In any case, I've implemented them all (note that this one that doesnt actually work in GAS - https://issuetracker.google.com/issues/423737982)
+
+I mainly use them when emulating Apps Script SpreadsheetApp services too as a double check that they are working as intended, but sometimes I build the requests up from scratch if it makes the automation simpler.
+
+If you want to see how these are all generated, see the constructor in services/advsheets/fakeadvsheets.
+
+#### Handling multiple response variations and formats.
+
+If you retrieve a cell format that has been set in the UI (or in Apps Script), you often get a less full response than one that has been set using the API. If you are using the Advanced Sheets Service, and you ask for "numberFormat" for example, you may get just the pattern (0.###) or you may get the full cellformat data { type: "NUMBER", pattern: "0.###""}. You'll have to be ready to handle either type of response depending on how (and perhaps even when) the value was originally created. This could apply to any fetches of format values. 
+
+Something like this should do the trick.
+````js
+const extractPattern = (response) => {
+  // a plain pattern entered by UI, apps script or lax api call
+  if (is.string(response)) return response
+  // should be { type: "TYPE", pattern: "xxx"}
+  if (!is.object(response) || !Reflect.has(response, "pattern")) return null
+  return response.pattern
+}
+````
+
+To emulate the regular SpreadsheetApp behavior, `fakeRange.getNumberFormat()` will strip out any extra stuff and just return the pattern. `fakeRange.setNumberFormat("0.###")` will always set the complete cellformat object { type: "NUMBER", pattern: "0.###"}
+
+##### Numberformat default pattern
+
+Normally we can use a null value to reset a format to the default UI value. However, number format will fail messily with a null argument. The correct way is `setNumberFormat('general')` even though `getNumberFormat()` returns '0.###############" or similar. If using Advanced Sheets, you still need to use the 'pattern' approach - { pattern: "general", type: "NUMBER" }
+
+#### Text direction 
+
+Unlike other similar functions, `setTextDirection(TextDirection)` takes an enum argument and `getTextDirection()` returns an enum too. `setTextDirection(null)` will reset to default behavior, but a subsequent  `getTextDirection()` will return null, rather than a default value. This allows the Sheets UI to make an in context decision based on language locale.
+
+#### Horizontal alignment
+
+The documented acceptable values to `range.setHorizontalAlignment()` are left, center, normal, null. However right is also valid so I'm supporting that too. `range.getHorizontalAlignment()` returns left,center,right,general,general-left. Although the alignment behavior for 'general' and 'general-left' in the UI appears identical, `range.setHorizontalAlignment(null)` returns 'general', whereas  `range.setHorizontalAlignment('normal')` returns 'general-left'. There doesn't appear to be a way to force a 'general-left' return via the Sheets API or advanced service.
+
+As with most of these format setting methods, Apps Script will silently ignore invalid arguments. I've generally throw an error if an invalid value argument is sent so, by design, `range.setHorizontalAlignment('foo') will throw an error on FakeGas, but not on Apps Script.
+
+#### Wrap and Wrap strategy
+
+Initially a cell will return OVERFLOW for `getWrapStrategy` and true for `getWrap`. This is wrong as OVERFLOW should be paired with false. Once you set wrapStrategy explicitly to OVERFLOW, it returns the correct value of false.
+
+The Apps Script issue for that is here https://issuetracker.google.com/issues/427134600
+
+#### range.copyValuesToRange
+
+The documentaton for this method says - "Copy the content of the range to the given location. If the destination is larger or smaller than the source range then the source is repeated or truncated accordingly."
+
+This implies that a smaller destination range that the source should only paste a truncated version of the source range. In fact it pastes it all - see issue https://issuetracker.google.com/issues/427192537
+
+I'm pausing implementation on this one till I see what I should actually implement
+
+
+
+#### TextRotation
+
+
+Apps Script returns a `TextRotation` object to `range.getTextRotation()`, which has both an 'isVertical()' and `getDegrees()` method. There is an overload for the `setTextRotation(degrees)` function - `setTextRotation(TextRotation)` which theoretically allows you to set a vertical or and angle. https://developers.google.com/apps-script/reference/spreadsheet/range#settextrotationrotation
+
+However, unlike most objects like this, there is not a `SpreadsheetApp.newTextRotation()`, and the object returned by `getTextRotation()` is readonly with no set variants. Trying to pass a plain JavaScript object with the assumed properties results in this error.
+
+````
+Exception: The parameters ((class)) don't match the method signature for SpreadsheetApp.Range.setTextRotation.
+````
+So the conclusion is that the overload for `setTextRotation(TextRotation)` does not work, so I won't be implementing this until the issue is resolved. `setTextRotation(degrees)` has been implemented of course.
+
+[See this issue for more information ](https://issuetracker.google.com/issues/425390984)
+
+
+Here's Gemini's verdict on textRotation
+
+"You are absolutely right, and I sincerely apologize once again for the continuous string of incorrect information regarding SpreadsheetApp's TextRotation capabilities. This specific part of the Apps Script API is surprisingly complex and poorly documented/intuitive."
+
+
+There's also a bug in the advanced sheet service - it doesn't return an angle in its response, even though it is set in the UI and even though Range.getTextRotation() correctly returns the angle. See https://issuetracker.google.com/issues/425390984.
+
+Since I'm using the API I can't detect the angle until that issue is fixed, so an angle set by the UI will always be seen as 0.
+
+
+#### Dates and sheets advanced service
+
+Dates can be stored in 'Excel dateserial' format in the API. This is a float showing how many days have passed since the Excel epoch which was Dec 30th, 1899. Here's a function to convert JS dates to that, which may be helpful if you are using the sheets advanced service, rather than the SpreadsheetApp service.
+````js
+const dateToSerial = (date) => {
+  if (!is.date(date)) {
+    throw new Error(`dateToSerial is expecting a date but got ${is(date)}`)
+  }
+  // these are held in a serial number like in Excel, rather than JavaScript epoch
+  // so the epoch is actually Dec 30 1899 rather than Jan 1 1970
+  const epochCorrection = 2209161600000
+  const msPerDay = 24 * 60 * 60 * 1000
+  const adjustedMs = date.getTime() + epochCorrection
+  return adjustedMs / msPerDay
+}
+````
+To enter this, you submit do this to create the value for your updateCells request body.
+````js
+const value = Sheets.newExtendedValue().setNumberValue(dateToSerial(value))
+````
+Note that this simply enters the numeric value of the dateSerial, without mentioning that it actually a date. To fix it as a date, you'll need to follow up with an userEnterFormat request to set the type to a date along with a custom format if required.
+
 #### UI settings
 
 Some of the options available in the GAS UI for setting or examining data validation are not available via GAS, and may not be available via Sheets. I'll update that later once I've figured the exact omissions and dicovered if there's a workaround. Since I'm implementing what GAS can currently do, not what it should do, this may not be an issue - just disappointing omissions.
@@ -362,6 +492,9 @@ Despite the various defaults, a missing value for these properties returned via
 
 All Apps Script enums are imitated using a seperate class 'newFakeGasenum()'. A complete write up of that is in [fakegasenum](https://github.com/brucemcpherson/fakegasenum). The same functionality is also available as an Apps Script library if you'd like to make your own enums over on GAS just like you find in Apps Script.
 
+## Auth
+Sometime between v144 and v150 of googleapis library, it appeared to become mandatory to include the project id in the auth pattern for API clients. Since we get the project id from the ADC, we actually have to do double auths. One to get the project id (which is async), and another to get an auth with the scopes required for the sheets, drive etc client (which is not async). All this now taken care of during the init phase, so look at an existing getauthenticated client function  for how if you are adding a new service,
+
 ## Help
 
 As I mentioned earlier, to take this further, I'm going to need a lot of help to extend the methods and services supported - so if you feel this would be useful to you, and would like to collaborate, please ping me on [bruce@mcpher.com](mailto:bruce@mcpher.com) and we'll talk.
@@ -369,4 +502,4 @@ As I mentioned earlier, to take this further, I'm going to need a lot of help to
 ## Translations and writeups
 
 - [mcpher](https://ramblings.mcpher.com/a-proof-of-concept-implementation-of-apps-script-environment-on-node/)
-- [Russian version](README.RU.md) ([credit Alex Ivanov](https://github.com/oshliaer))
+- [Russian version](README.RU.md) ([credit Alex Ivanov](https://github.com/oshliaer)) - needs updating

package/package.json

@@ -7,15 +7,14 @@
     "@sindresorhus/is": "^7.0.1",
     "archiver": "^7.0.1",
     "get-stream": "^9.0.1",
-    "google-auth-library": "^9.15.0",
-    "googleapis": "^144.0.0",
+    "googleapis": "^150.0.1",
     "got": "^14.4.5",
     "into-stream": "^8.0.1",
     "keyv": "^5.2.3",
     "keyv-file": "^5.1.1",
-    "make-synchronous": "^1.0.0",
     "mime": "^4.0.6",
     "sleep-synchronously": "^2.0.0",
+    "subsume": "^4.0.0",
     "to-readable-stream": "^4.0.0",
     "unzipper": "^0.12.3"
   },
@@ -34,10 +33,12 @@
     "testscriptapp": "cp mainlocal.js main.js &&  node --env-file=.env ./test/testscriptapp.js execute",
     "testfiddler": "cp mainlocal.js main.js &&  node --env-file=.env ./test/testfiddler.js execute",
     "testenums": "cp mainlocal.js main.js &&  node --env-file=.env ./test/testenums.js execute",
+    "testsheetssets": "cp mainlocal.js main.js &&  node --env-file=.env ./test/testsheetssets.js execute",
+    "testsheetsvui": "cp mainlocal.js main.js &&  node --env-file=.env ./test/testsheetsvui.js execute",
     "pub": "cp mainlocal.js main.js && npm publish --access public"
   },
   "name": "@mcpher/gas-fakes",
-  "version": "1.0.9",
+  "version": "1.0.10",
   "main": "main.js",
   "description": "A proof of concept implementation of Apps Script Environment on Node",
   "repository": "github:brucemcpherson/gas-fakes",