@ferchy/n8n-nodes-aimc-toolkit 0.1.18 → 0.1.21

package/README.md

@@ -4,7 +4,7 @@ AIMC Toolkit is a community node package for n8n with focused nodes:
 
 - **AIMC Code**: run JavaScript with a practical toolbox of libraries.
 - **AIMC Media**: FFmpeg-powered media operations without extra glue nodes.
-- **AIMC Social Scraper**: scrape content from social media platforms (TikTok, Instagram, etc.). Note: YouTube is available via AIMC Code node.
+- **AIMC Social Scraper**: scrape content from social media platforms (coming soon).
 
 ## Why I Built This
 
@@ -23,7 +23,7 @@ n8n is powerful, but real workflows often need basic utilities (validation, pars
 
 - **Fewer nodes, cleaner flows**: consolidate multiple steps into one code node.
 - **Media ready**: convert, compress, merge, and inspect media in one place.
-- **Social scraping**: scrape TikTok, Instagram, and more with a simple UI.
+- **Social scraping**: scrape social media platforms with a simple UI (coming soon).
 - **Practical libraries**: parsing, validation, dates, and web utilities built in.
 
 ## Installation
@@ -113,7 +113,7 @@ Inputs: items are available as `items` (all) or `item` (single)
 Return: return a plain object or array of objects (JSON-safe)
 Libraries (globals): axios, lodash (_), zod (z), joi (Joi), yup, Ajv, validator, dayjs, dateFns, dateFnsTz, moment,
 cheerio, papaparse (Papa), yaml (YAML), xml2js, XMLParser, qs, FormData, uuid, nanoid, bytes, cronParser, ms,
-fuzzy, stringSimilarity, slug, pluralize, jsonDiff, htmlToText, marked, qrcode/QRCode, ytdl, tiktokScraper/TikTokScraper, ffmpeg, ffmpegStatic, ffprobeStatic.
+fuzzy, stringSimilarity, slug, pluralize, jsonDiff, htmlToText, marked, qrcode/QRCode, ytdl, ffmpeg, ffmpegStatic, ffprobeStatic.
 Helpers: utils.now(), utils.safeJson(), utils.toArray()
 Please return only the JavaScript code for the AIMC Code node.
 ```
@@ -193,85 +193,12 @@ Use **Input Mode = File Path** to avoid loading big files into memory.
 
 ### AIMC Social Scraper
 
-> **💡 Tip**: You can also use TikTok scraper programmatically in the **AIMC Code** node using `tiktokScraper` or `TikTokScraper` global variables. This UI node provides a simpler interface for common scraping tasks.
+**Coming Soon!**
 
-> **⚠️ Important Legal Notice**: This node uses web scraping techniques, not official APIs. It is provided for educational and personal use only. Users are responsible for complying with:
-> - TikTok's Terms of Service
-> - Applicable laws and regulations in their jurisdiction
-> - Rate limiting and respectful usage practices
-> - Copyright and intellectual property rights
->
-> **This is not an official API support.** This scraper uses TikTok's web interface to extract media and metadata. Use at your own risk. The authors and maintainers are not responsible for any misuse or violations of terms of service.
-
-**Platforms**
-- **TikTok** (fully supported)
-- Instagram, Twitter/X (coming soon)
+Social media scraping functionality is currently under development. This node will support scraping content from various platforms including TikTok, Instagram, Twitter/X, and more.
 
 **Note**: YouTube support is already available via the **AIMC Code** node using the `ytdl` library. See the Media Helpers section for details.
 
-**TikTok Scrape Types**
-- **User Posts**: Scrape posts from a specific user
-- **Hashtag**: Scrape posts from a hashtag
-- **Trending**: Get trending posts
-- **Single Video**: Get metadata for a specific video
-
-**Key Options**
-- **Number of Posts**: How many posts to scrape (1-1000)
-- **Options** (collapsible section):
-  - **No Watermark**: Download videos without TikTok watermark
-  - **HD Video**: Download in HD quality
-  - **By User ID**: Treat username as user ID (for large IDs)
-  - **Session Cookie** (Advanced): Only needed if you encounter rate limits - see instructions below
-- **No Watermark**: Download videos without TikTok watermark
-- **HD Video**: Download in HD quality
-- **By User ID**: Treat username as user ID (for large IDs)
-
-**Example: Scrape User Posts**
-```
-Platform: TikTok
-Scrape Type: User Posts
-Username: username
-Number of Posts: 50
-Options (expand if needed):
-  - No Watermark: true
-  - Session Cookie: (only if you hit rate limits)
-```
-
-**Example: Scrape Hashtag**
-```
-Platform: TikTok
-Scrape Type: Hashtag
-Hashtag: fyp
-Number of Posts: 100
-```
-
-**Example: Get Trending**
-```
-Platform: TikTok
-Scrape Type: Trending
-Number of Posts: 30
-```
-
-**Session Cookie (Advanced - Optional)**
-The scraper works **without cookies by default**. Session cookies are only needed if you encounter rate limits. They're hidden in the **Options** section - you don't need to worry about them unless you have issues.
-
-**When you might need a session cookie:**
-- You're scraping many posts and getting rate limited
-- You're seeing captcha challenges
-- You need to scrape large amounts of data regularly
-
-**To get a session cookie (only if needed):**
-1. Open TikTok in your browser and log in
-2. Open DevTools (F12 or Right-click > Inspect)
-3. Go to **Application** tab > **Cookies** > `https://www.tiktok.com`
-4. Find the `sid_tt` cookie
-5. Copy the entire value (e.g., `sid_tt=abc123def456...`)
-6. Expand **Options** in the node and paste into the Session Cookie field
-
-**Tip**: Start without a cookie. Most users won't need it. Only add one if you hit rate limits.
-
-**Note**: Session cookies help avoid rate limits and captchas. They're optional but recommended for production use.
-
 ## Library Reference (AIMC Code)
 
 Libraries are available as globals or via `libs.<name>`.
@@ -344,7 +271,6 @@ Provide Python code and set a variable named `result` to return output.
 ### Media Helpers
 - `qrcode` (global `QRCode` or `qrcode`): generate QR codes as data URLs or images.
 - `@distube/ytdl-core` (global `ytdl`): download media from supported sources. Use for media ingestion flows.
-- `tiktok-scraper` (global `tiktokScraper` or `TikTokScraper`): scrape TikTok content including users, hashtags, trending topics, videos, and metadata. Supports downloading videos (with/without watermarks), session cookies, and proxies.
 - `fluent-ffmpeg` (global `ffmpeg`): build FFmpeg pipelines in JS when you need custom media logic.
 - `ffmpeg-static` (global `ffmpegStatic`): path to a bundled FFmpeg binary (if available).
 - `ffprobe-static` (global `ffprobeStatic`): path to a bundled ffprobe binary (if available).
@@ -382,7 +308,7 @@ Environment variables:
 The AIMC Social Scraper node uses web scraping techniques to extract data from social media platforms. This is **not** an official API integration and is provided for educational and personal use only.
 
 **Important Points:**
-- This scraper uses the web interface of platforms (TikTok, etc.), not official APIs
+- This scraper uses the web interface of platforms, not official APIs
 - Users must comply with each platform's Terms of Service
 - Users must comply with applicable laws and regulations (GDPR, CCPA, etc.)
 - Users are responsible for respecting rate limits and avoiding abusive behavior

package/dist/nodes/AimcSocial/AimcSocial.node.js

@@ -1,55 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.AimcSocial = void 0;
-const n8n_workflow_1 = require("n8n-workflow");
-let tiktokScraper = null;
-function getTikTokScraper() {
-    if (!tiktokScraper) {
-        try {
-            tiktokScraper = require('tiktok-scraper');
-        }
-        catch (error) {
-            throw new Error('TikTok scraper not available. Please ensure tiktok-scraper is installed.');
-        }
-    }
-    return tiktokScraper;
-}
-async function scrapeTikTokUser(username, options) {
-    const scraper = getTikTokScraper();
-    return await scraper.user(username, {
-        number: options.number || 20,
-        sessionList: options.sessionList || [],
-        by_user_id: options.byUserId || false,
-        noWaterMark: options.noWaterMark || false,
-        hdVideo: options.hdVideo || false,
-    });
-}
-async function scrapeTikTokHashtag(hashtag, options) {
-    const scraper = getTikTokScraper();
-    return await scraper.hashtag(hashtag, {
-        number: options.number || 20,
-        sessionList: options.sessionList || [],
-        noWaterMark: options.noWaterMark || false,
-        hdVideo: options.hdVideo || false,
-    });
-}
-async function scrapeTikTokTrend(options) {
-    const scraper = getTikTokScraper();
-    return await scraper.trend('', {
-        number: options.number || 20,
-        sessionList: options.sessionList || [],
-        noWaterMark: options.noWaterMark || false,
-        hdVideo: options.hdVideo || false,
-    });
-}
-async function scrapeTikTokVideo(videoUrl, options) {
-    const scraper = getTikTokScraper();
-    return await scraper.video(videoUrl, {
-        sessionList: options.sessionList || [],
-        noWaterMark: options.noWaterMark || false,
-        hdVideo: options.hdVideo || false,
-    });
-}
 class AimcSocial {
     constructor() {
         this.description = {
@@ -58,7 +9,7 @@ class AimcSocial {
             icon: 'file:aimc-social.svg',
             group: ['transform'],
             version: 1,
-            description: 'Scrape content from social media platforms (TikTok, Instagram, etc.). Note: YouTube is available via AIMC Code node using ytdl.',
+            description: 'Scrape content from social media platforms. Coming soon!',
             defaults: {
                 name: 'AIMC Social Scraper',
             },
@@ -75,133 +26,7 @@ class AimcSocial {
                         { name: 'Twitter/X', value: 'twitter' },
                     ],
                     default: 'tiktok',
-                    description: 'Select the social media platform to scrape. Note: YouTube is available via AIMC Code node using the ytdl library.',
-                },
-                {
-                    displayName: 'Scrape Type',
-                    name: 'scrapeType',
-                    type: 'options',
-                    options: [
-                        { name: 'User Posts', value: 'user' },
-                        { name: 'Hashtag', value: 'hashtag' },
-                        { name: 'Trending', value: 'trending' },
-                        { name: 'Single Video', value: 'video' },
-                    ],
-                    default: 'user',
-                    displayOptions: {
-                        show: {
-                            platform: ['tiktok'],
-                        },
-                    },
-                    description: 'What type of content to scrape',
-                },
-                {
-                    displayName: 'Username',
-                    name: 'username',
-                    type: 'string',
-                    default: '',
-                    required: true,
-                    displayOptions: {
-                        show: {
-                            platform: ['tiktok'],
-                            scrapeType: ['user'],
-                        },
-                    },
-                    placeholder: 'username or @username',
-                    description: 'TikTok username (with or without @ symbol - will be automatically removed)',
-                },
-                {
-                    displayName: 'Hashtag',
-                    name: 'hashtag',
-                    type: 'string',
-                    default: '',
-                    required: true,
-                    displayOptions: {
-                        show: {
-                            platform: ['tiktok'],
-                            scrapeType: ['hashtag'],
-                        },
-                    },
-                    placeholder: 'fyp or #fyp',
-                    description: 'Hashtag to scrape (with or without # symbol - will be automatically removed)',
-                },
-                {
-                    displayName: 'Video URL',
-                    name: 'videoUrl',
-                    type: 'string',
-                    default: '',
-                    required: true,
-                    displayOptions: {
-                        show: {
-                            platform: ['tiktok'],
-                            scrapeType: ['video'],
-                        },
-                    },
-                    placeholder: 'https://www.tiktok.com/@username/video/1234567890',
-                    description: 'Full URL of the TikTok video',
-                },
-                {
-                    displayName: 'Number of Posts',
-                    name: 'number',
-                    type: 'number',
-                    default: 20,
-                    typeOptions: {
-                        minValue: 1,
-                        maxValue: 1000,
-                    },
-                    displayOptions: {
-                        show: {
-                            platform: ['tiktok'],
-                            scrapeType: ['user', 'hashtag', 'trending'],
-                        },
-                    },
-                    description: 'Number of posts to scrape',
-                },
-                {
-                    displayName: 'Options',
-                    name: 'options',
-                    type: 'collection',
-                    placeholder: 'Add Option',
-                    default: {},
-                    displayOptions: {
-                        show: {
-                            platform: ['tiktok'],
-                        },
-                    },
-                    options: [
-                        {
-                            displayName: 'No Watermark',
-                            name: 'noWaterMark',
-                            type: 'boolean',
-                            default: false,
-                            description: 'Whether to download videos without watermark',
-                        },
-                        {
-                            displayName: 'HD Video',
-                            name: 'hdVideo',
-                            type: 'boolean',
-                            default: false,
-                            description: 'Whether to download HD quality videos',
-                        },
-                        {
-                            displayName: 'By User ID',
-                            name: 'byUserId',
-                            type: 'boolean',
-                            default: false,
-                            description: 'If true, interpret the username as a user ID (only applies to User Posts scrape type)',
-                        },
-                        {
-                            displayName: 'Session Cookie',
-                            name: 'sessionCookie',
-                            type: 'string',
-                            typeOptions: {
-                                password: true,
-                            },
-                            default: '',
-                            placeholder: 'sid_tt=abc123...',
-                            description: '[Advanced] TikTok session cookie to reduce rate limits. Only needed if you encounter rate limits. Leave empty to use without. Format: "sid_tt=abc123def456..."',
-                        },
-                    ],
+                    description: 'Select the social media platform to scrape',
                 },
             ],
         };
@@ -210,84 +35,15 @@ class AimcSocial {
         const items = this.getInputData();
         const returnData = [];
         for (let i = 0; i < items.length; i++) {
-            try {
-                const platform = this.getNodeParameter('platform', i);
-                const scrapeType = this.getNodeParameter('scrapeType', i);
-                const options = this.getNodeParameter('options', i, {});
-                let result;
-                if (platform === 'tiktok') {
-                    // Get session cookie from options if provided
-                    const sessionCookie = options.sessionCookie || '';
-                    const sessionList = sessionCookie ? [sessionCookie] : [];
-                    if (scrapeType === 'user') {
-                        const usernameRaw = this.getNodeParameter('username', i);
-                        if (!usernameRaw || !usernameRaw.trim()) {
-                            throw new n8n_workflow_1.NodeOperationError(this.getNode(), 'Username is required. Please enter a TikTok username (without @ symbol).');
-                        }
-                        // Remove @ symbol if present and trim whitespace
-                        const username = usernameRaw.trim().replace(/^@/, '');
-                        const number = this.getNodeParameter('number', i, 20);
-                        result = await scrapeTikTokUser.call(this, username, {
-                            number,
-                            sessionList,
-                            byUserId: options.byUserId,
-                            noWaterMark: options.noWaterMark,
-                            hdVideo: options.hdVideo,
-                        });
-                    }
-                    else if (scrapeType === 'hashtag') {
-                        const hashtagRaw = this.getNodeParameter('hashtag', i);
-                        if (!hashtagRaw || !hashtagRaw.trim()) {
-                            throw new n8n_workflow_1.NodeOperationError(this.getNode(), 'Hashtag is required. Please enter a TikTok hashtag (without # symbol).');
-                        }
-                        // Remove # symbol if present and trim whitespace
-                        const hashtag = hashtagRaw.trim().replace(/^#/, '');
-                        const number = this.getNodeParameter('number', i, 20);
-                        result = await scrapeTikTokHashtag.call(this, hashtag, {
-                            number,
-                            sessionList,
-                            noWaterMark: options.noWaterMark,
-                            hdVideo: options.hdVideo,
-                        });
-                    }
-                    else if (scrapeType === 'trending') {
-                        const number = this.getNodeParameter('number', i, 20);
-                        result = await scrapeTikTokTrend.call(this, {
-                            number,
-                            sessionList,
-                            noWaterMark: options.noWaterMark,
-                            hdVideo: options.hdVideo,
-                        });
-                    }
-                    else if (scrapeType === 'video') {
-                        const videoUrl = this.getNodeParameter('videoUrl', i);
-                        result = await scrapeTikTokVideo.call(this, videoUrl, {
-                            sessionList,
-                            noWaterMark: options.noWaterMark,
-                            hdVideo: options.hdVideo,
-                        });
-                    }
-                }
-                else {
-                    throw new n8n_workflow_1.NodeOperationError(this.getNode(), `Platform "${platform}" is not yet implemented. Currently only TikTok is supported.`);
-                }
-                returnData.push({
-                    json: result,
-                    pairedItem: { item: i },
-                });
-            }
-            catch (error) {
-                if (this.continueOnFail()) {
-                    returnData.push({
-                        json: {
-                            error: error instanceof Error ? error.message : String(error),
-                        },
-                        pairedItem: { item: i },
-                    });
-                    continue;
-                }
-                throw error;
-            }
+            const platform = this.getNodeParameter('platform', i);
+            returnData.push({
+                json: {
+                    message: 'AIMC Social Scraper is coming soon!',
+                    platform: platform,
+                    note: 'This feature is currently under development. Stay tuned for updates!',
+                },
+                pairedItem: { item: i },
+            });
         }
         return [returnData];
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ferchy/n8n-nodes-aimc-toolkit",
-  "version": "0.1.18",
+  "version": "0.1.21",
   "description": "AIMC Toolkit nodes for n8n: code execution and media operations.",
   "license": "MIT",
   "author": "Ferchy",
@@ -88,7 +88,6 @@
     "uuid": "^11.1.0",
     "validator": "^13.11.0",
     "xml2js": "^0.6.2",
-    "tiktok-scraper": "git+https://github.com/drawrowfly/tiktok-scraper.git",
     "yaml": "^2.8.1",
     "yup": "^1.7.0",
     "zod": "^3.25.76"