uchr 0.3.0__py3-none-any.whl

uchr-0.3.0.dist-info/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Miki Yutani
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

uchr-0.3.0.dist-info/METADATA

@@ -0,0 +1,314 @@
+Metadata-Version: 2.3
+Name: uchr
+Version: 0.3.0
+Summary: Unicode character tools
+License: MIT
+Keywords: unicode,character,tools,cli
+Author: Miki Yutani
+Author-email: mkyutani@gmail.com
+Requires-Python: >=3.8,<4.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing :: General
+Classifier: Topic :: Utilities
+Requires-Dist: requests (>=2.25.0,<3.0.0)
+Project-URL: Homepage, https://github.com/mkyutani/unicode-tools
+Project-URL: Repository, https://github.com/mkyutani/unicode-tools
+Description-Content-Type: text/markdown
+
+# Unicode Tools
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![Unicode](https://img.shields.io/badge/Unicode-15.0-green.svg)](https://unicode.org/)
+
+A powerful command-line tool for searching and exploring Unicode characters, emoji sequences, and character properties.
+
+## 🚀 Features
+
+- **Search by name**: Find characters by their Unicode name
+- **Search by code**: Look up characters by code point or range
+- **Search by character**: Reverse lookup from character to details
+- **Search by block**: Explore characters within Unicode blocks
+- **Emoji support**: Full support for emoji sequences and ZWJ sequences
+- **CJK details**: Enhanced descriptions for CJK characters using kDefinition
+- **Flexible output**: Multiple output formats for different use cases
+
+## 📖 Table of Contents
+
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Usage Examples](#usage-examples)
+- [Command Reference](#command-reference)
+- [Database Management](#database-management)
+- [Contributing](#contributing)
+- [License](#license)
+
+## 🛠 Installation
+
+### Install from source
+
+```bash
+git clone https://github.com/mkyutani/unicode-tools.git
+cd unicode-tools
+pip install -e .
+```
+
+### Initialize database
+
+Create the Unicode database (required for first use):
+
+```bash
+uchr db create
+```
+
+This downloads Unicode 15.0 data and creates a local SQLite database (~13MB) at:
+- Linux/macOS: `~/.local/share/unicode-tools/unicode.db`
+- Root users: Automatically chooses between system (`/var/lib/unicode-tools/`) or personal location
+
+## ⚡ Quick Start
+
+```bash
+# Search for ghost-related characters
+uchr search ghost
+
+# Find characters in a code range
+uchr search -c 1F47A-1F480
+
+# Search by character
+uchr search -x 👻
+
+# Search within a Unicode block
+uchr search -b "Emoticons"
+```
+
+## 📋 Usage Examples
+
+### Search by Name
+
+```bash
+uchr search goblin
+```
+```
+👺 1F47A JAPANESE GOBLIN
+```
+
+### Search by Code Range
+
+```bash
+uchr search -c 1F479-1F47B
+```
+```
+👹 1F479 JAPANESE OGRE
+👺 1F47A JAPANESE GOBLIN
+👻 1F47B GHOST
+```
+
+### Search by Character
+
+```bash
+uchr search -x 👻
+```
+```
+👻 1F47B GHOST
+```
+
+### Search by Unicode Block
+
+```bash
+uchr search -b "Misc_Pictographs"
+```
+
+### Search with Details (CJK Characters)
+
+```bash
+uchr search -d "pray for happiness"
+```
+```
+祝 795D CJK UNIFIED IDEOGRAPH-#; PRAY FOR HAPPINESS OR BLESSINGS
+```
+
+### Output Formatting
+
+```bash
+# Simple format (characters only)
+uchr search ghost -f simple
+👻
+
+# UTF-8 format
+uchr search ghost -f utf8
+👻 F0 9F 91 BB GHOST
+
+# Custom delimiter
+uchr search ghost -D "|"
+👻|1F47B|GHOST
+```
+
+## 🔧 Command Reference
+
+### uchr
+
+Main command with subcommands for all Unicode operations.
+
+#### uchr search
+
+Search Unicode characters with various criteria.
+
+| Option | Short | Description |
+|--------|-------|-------------|
+| `--name` | | Search by character name (default) |
+| `--code` | `-c` | Search by code point or range |
+| `--char` | `-x` | Search by character |
+| `--block` | `-b` | Search by Unicode block |
+| `--detail` | `-d` | Search in character details |
+| `--strict` | `-s` | Exact match (case insensitive) |
+| `--first` | `-1` | Show first result only |
+| `--format` | `-f` | Output format: `utf8`, `simple` |
+| `--delimiter` | `-D` | Custom delimiter (default: space) |
+
+#### uchr db
+
+Database management operations.
+
+| Subcommand | Description |
+|------------|-------------|
+| `uchr db create` | Create/update Unicode database |
+| `uchr db delete` | Remove Unicode database |
+| `uchr db info` | Show database location |
+
+## 💾 Database Management
+
+### Create Database
+```bash
+uchr db create
+```
+
+### Check Database Location
+```bash
+uchr db info
+```
+
+### Remove Database
+```bash
+uchr db delete
+```
+
+### Environment Variables
+
+- `UNICODE_DB_PATH`: Override default database location
+
+```bash
+export UNICODE_DB_PATH="/custom/path/unicode.db"
+uchr db create
+```
+
+## 🌟 Advanced Examples
+
+### Finding Emoji Sequences
+
+```bash
+# National flags
+uchr search -b "RGI_Emoji_Flag_Sequence"
+
+# Family emoji with ZWJ sequences
+uchr search family
+```
+
+### Terminal Display vs. Browser/Application Support
+
+Many terminals don't properly display complex emoji sequences, but the characters work correctly when copied to browsers or applications.
+
+#### National Flag Example
+
+When searching for flags, you might see separate letters in your terminal:
+
+```bash
+uchr search -b "RGI_Emoji_Flag_Sequence" | grep -i norway
+```
+```
+🇳🇴 1F1F3 1F1F4 flag: Norway
+```
+
+![Sample to copy Norway's flag in twitter](img/ucsearch-block-flag-norway.png)
+
+Even though you see two separate letters (🇳🇴) in the terminal, when you copy and paste them into a browser or application like Twitter, they combine to display the Norwegian flag 🇳🇴.
+
+![Sample to paste Norway's flag in twitter](img/twitter-norway.png)
+
+#### ZWJ Sequence Example
+
+The same applies to Zero Width Joiner (ZWJ) sequences. Complex emoji like family groups or professional emoji might not render correctly in terminals:
+
+```bash
+uchr search "polar bear"
+```
+
+In a terminal without proper font support:
+
+![Sample to copy polar bear in twitter](img/ucsearch-polarbear.png)
+
+But when pasted in Twitter or other applications:
+
+![Sample to paste polar bear in twitter](img/twitter-polarbear.png)
+
+> **💡 Tip**: This is expected behavior. The Unicode data is correct, and the characters will work properly in applications that support modern emoji rendering.
+
+### Pipe Operations
+
+```bash
+# Get just the character
+uchr search ghost -f simple
+
+# First match only
+uchr search snow -1
+
+# Custom format for scripting
+uchr search ghost -D "," | cut -d',' -f1
+```
+
+### Complex Searches
+
+```bash
+# CJK characters with specific meanings
+uchr search -d "dragon"
+
+# Characters in multiple blocks
+uchr search -b "Mathematical" | head -10
+```
+
+## 🏗 Data Sources
+
+This tool uses official Unicode 15.0 data:
+
+- [Unicode Character Database](https://www.unicode.org/Public/15.0.0/ucdxml/ucd.all.flat.zip)
+- [Emoji Sequences](https://www.unicode.org/Public/emoji/15.0/emoji-sequences.txt)
+- [Emoji ZWJ Sequences](https://www.unicode.org/Public/emoji/15.0/emoji-zwj-sequences.txt)
+
+## 🤝 Contributing
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## 🙏 Acknowledgments
+
+- [Unicode Consortium](https://unicode.org/) for maintaining Unicode standards
+- Contributors and users of this project
+

uchr-0.3.0.dist-info/RECORD

@@ -0,0 +1,11 @@
+unicode_tools/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+unicode_tools/database.py,sha256=WqPBF7Rmyq74ONv3awcuUsSQkB5hGJZ3MB2v7rbQsck,11329
+unicode_tools/db.py,sha256=G2y13GA_XdoV13Lmy-wcPEQ93-QdU03tWRrL6ALYyhs,4463
+unicode_tools/normalize.py,sha256=ZeybMGBIWTaQjjyrg75DNsqyNwM_4p4SAqn2Wfm23QU,3560
+unicode_tools/search.py,sha256=ovhGPTFzXuRs0YX_ljhzttTqw5IC20WAtOcwEexn4DM,2636
+unicode_tools/uchr.py,sha256=qAutAuD8hNnMnhpulxveYkiIVTcvEzMqQg6O3OPFNQQ,6807
+uchr-0.3.0.dist-info/LICENSE,sha256=9btWvx8-OK73XlFie43RPuK8tWhKKu_IcEjqtVQjC-c,1068
+uchr-0.3.0.dist-info/METADATA,sha256=5ddxWmIyOvFMbNZBF8cu8vrFAimPbKMSkxz5dtVSBhc,7977
+uchr-0.3.0.dist-info/WHEEL,sha256=fGIA9gx4Qxk2KDKeNJCbOEwSrmLtjWCwzBz351GyrPQ,88
+uchr-0.3.0.dist-info/entry_points.txt,sha256=l5mV9P_tkP_q8Z4_rGP1CbP7vV2dyMTos6C-jwzkR3c,48
+uchr-0.3.0.dist-info/RECORD,,

uchr-0.3.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: poetry-core 2.1.2
+Root-Is-Purelib: true
+Tag: py3-none-any

uchr-0.3.0.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+uchr=unicode_tools.uchr:uchr
+

unicode_tools/database.py

@@ -0,0 +1,331 @@
+#!/usr/bin/env python3
+
+import io
+import os
+import re
+import sqlite3
+import sys
+import tempfile
+import xml.etree.ElementTree as et
+import zipfile
+from pathlib import Path
+from urllib.parse import urlparse
+
+import requests
+
+from .db import AutoID, Connection, Cursor, Database
+
+namespace = "{http://www.unicode.org/ns/2003/ucd/1.0}"
+tag_ucd = namespace + "ucd"
+tag_description = namespace + "description"
+tag_repertoire = namespace + "repertoire"
+tag_char = namespace + "char"
+tag_noncharacter = namespace + "noncharacter"
+tag_reserved = namespace + "reserved"
+tag_surrogate = namespace + "surrogate"
+tag_name_alias = namespace + "name-alias"
+
+table_char_autoincrement_id = AutoID().init()
+
+
+def download_ucd(ucd_zip_url):
+    ucd_zip_url_path = Path(urlparse(ucd_zip_url)[2])
+    ucd_zip_filename = ucd_zip_url_path.name
+    ucd_xml_filename = ucd_zip_url_path.with_suffix(".xml").name
+
+    zip_filepath = "(Not assigned)"
+
+    try:
+        with tempfile.TemporaryDirectory() as tmpdir:
+            zip_filepath = os.path.join(tmpdir, ucd_zip_filename)
+
+            print(f"Downloading {ucd_zip_url} ...", file=sys.stderr)
+
+            res = requests.get(ucd_zip_url, stream=True)
+            if res.status_code >= 400:
+                print(f"Fetch error: {res.status_code}", file=sys.stderr)
+                return None
+
+            content_type = res.headers["Content-Type"]
+            if content_type != "application/zip":
+                print(f"Invalid content type: {content_type}")
+                return None
+
+            with open(zip_filepath, "wb") as fzip:
+                for chunk in res.iter_content(chunk_size=1024):
+                    if chunk:
+                        fzip.write(chunk)
+                        fzip.flush()
+
+            print(f"Downloaded {zip_filepath}", file=sys.stderr)
+
+            with zipfile.ZipFile(zip_filepath, "r") as zip:
+                xml_list = zip.read(ucd_xml_filename)
+
+            print("Extracted unicode data xml", file=sys.stderr)
+
+            return xml_list
+
+    except Exception as e:
+        print(
+            f"Failed to download zip from {ucd_zip_url} to {zip_filepath}",
+            file=sys.stderr,
+        )
+        print(f"{type(e).__name__}: {str(e)}", file=sys.stderr)
+        return None
+
+
+def get_ucd_cp(tag):
+    cp = tag.attrib.get("cp")
+    first_cp = tag.attrib.get("first-cp")
+    last_cp = tag.attrib.get("last-cp")
+
+    if cp:
+        first_cp = cp
+        last_cp = cp
+
+    if not (first_cp and last_cp):
+        print(
+            f"Invalid code range: cp={cp}, first_cp={first_cp}, last_cp={last_cp}",
+            file=sys.stderr,
+        )
+        return None
+
+    min = int(first_cp, 16)
+    max = int(last_cp, 16)
+
+    return (min, max)
+
+
+def get_ucd_char_cp(char):
+    value = None
+    r = get_ucd_cp(char)
+    if r:
+        min = r[0]
+        max = r[1]
+        if char.tag != tag_char:
+            if min == max:
+                code_range = f"{min:X}"
+            else:
+                code_range = f"{min:X}-{max:X}"
+
+            if char.tag == tag_reserved:
+                print(f"Found reserved code(s): {code_range}", file=sys.stderr)
+            elif char.tag == tag_noncharacter:
+                print(f"Found non character code(s): {code_range}", file=sys.stderr)
+            elif char.tag == tag_surrogate:
+                print(f"Found surrogate code(s): {code_range}", file=sys.stderr)
+            else:
+                print(f"Found unknown tag: {char.tag} {code_range}", file=sys.stderr)
+            return []
+
+        value = range(min, max + 1)
+        return value
+
+
+def get_name(char):
+    value = []
+    name = char.attrib.get("na")
+    name1 = char.attrib.get("na1")
+    if name and len(name) > 0:
+        value.append(name)
+    if name1 and len(name1) > 0 and name != name1:
+        value.append(name1)
+    for alias in char:
+        if alias.tag == tag_name_alias:
+            alias_name = alias.attrib.get("alias")
+            if alias_name and len(alias_name) > 0 and alias_name not in value:
+                value.append(alias_name)
+    return "; ".join(value)
+
+
+def get_detail(char, name):
+    definition = char.attrib.get("kDefinition")
+    if definition and len(definition) > 0:
+        return "; ".join([name, definition.upper()])
+    else:
+        return name
+
+
+def store_ucd(xml_list):
+    if not xml_list:
+        return None
+
+    root = et.parse(io.BytesIO(xml_list)).getroot()
+    if root.tag != tag_ucd:
+        print(f"Unexpected XML scheme: {root.tag}", file=sys.stderr)
+        return None
+
+    repertoire = root.find(tag_repertoire)
+
+    with Connection() as conn:
+        with Cursor(conn) as cur:
+            count = 0
+            for char in repertoire:
+                code_range = get_ucd_char_cp(char)
+                for code in code_range:
+                    value_code = code
+                    value_code_text = f'"{value_code:X}"'
+                    name = get_name(char)
+                    if not name:
+                        print(f"Found no character: {code:X}", file=sys.stderr)
+                        continue
+                    value_name = f'"{name}"'
+                    detail = get_detail(char, name)
+                    value_detail = f'"{detail}"'
+
+                    try:
+                        if code == 0:
+                            value_char = "NULL"
+                        else:
+                            escaped_char = str(chr(code)).replace('"', '""')
+                            value_char = f'"{escaped_char}"'
+                    except ValueError:
+                        print(f"Invalid character {code:X} ({name})", file=sys.stderr)
+                        continue
+
+                    block = char.attrib.get("blk")
+                    if not block:
+                        print(f"No block name: {code:X}", file=sys.stderr)
+                        value_block = '"(None)"'
+                    else:
+                        value_block = f'"{block}"'
+
+                    id = table_char_autoincrement_id.next()
+                    dml = f"insert into char(id, name, detail, codetext, char, block) values({id}, {value_name}, {value_detail}, {value_code_text}, {value_char}, {value_block})"
+                    cur.execute(dml)
+                    dml_seq = f"insert into codepoint(char, seq, code) values({id}, 1, {value_code})"
+                    cur.execute(dml_seq)
+                    count = count + 1
+
+            conn.commit()
+            print(f"Stored {count} characters", file=sys.stderr)
+
+
+def download_emoji(emoji_txt_url):
+    try:
+        res = requests.get(emoji_txt_url)
+        if res.status_code >= 400:
+            print(f"Fetch error: {res.status_code}", file=sys.stderr)
+            return None
+
+        content_type = res.headers["Content-Type"]
+        if not ("text/plain" in content_type and "charset=utf-8" in content_type):
+            print(f"Invalid content type: {content_type}")
+            return None
+
+        return res.text.splitlines()
+
+    except Exception as e:
+        print(f"Failed to download emoji from {emoji_txt_url}", file=sys.stderr)
+        print(f"{type(e).__name__}: {str(e)}", file=sys.stderr)
+        return None
+
+
+def store_emoji(emoji_sequences):
+    if not emoji_sequences:
+        return
+
+    emoji_sequence_line_pattern = re.compile("^(.+);(.+);([^#]+)#")
+    emoji_sequence_cp_pattern = re.compile("([0-9A-Fa-f]+)")
+    emoji_sequence_multi_pattern = re.compile("([0-9A-Fa-f]+)")
+    emoji_sequence_continuous_pattern = re.compile(r"([0-9A-Fa-f]+)\.\.([0-9A-Fa-f]+)")
+
+    with Connection() as conn:
+        with Cursor(conn) as cur:
+            count = 0
+            for sequence in emoji_sequences:
+                if len(sequence) == 0 or sequence.startswith("#"):
+                    continue
+                emoji = emoji_sequence_line_pattern.match(sequence)
+                emoji_codes = emoji.group(1).strip()
+                emoji_type = emoji.group(2).strip()
+                emoji_name = emoji.group(3).strip()
+
+                cp_list = []
+                cp = re.fullmatch(emoji_sequence_cp_pattern, emoji_codes)
+                if cp:
+                    cp_list.append(int(emoji_codes, 16))
+                else:
+                    cp = re.match(emoji_sequence_continuous_pattern, emoji_codes)
+                    if cp:
+                        min = int(cp.group(1), 16)
+                        max = int(cp.group(2), 16)
+                        cp_list.extend(list(range(min, max + 1)))
+                    else:
+                        seq = []
+                        for cp in re.finditer(
+                            emoji_sequence_multi_pattern, emoji_codes
+                        ):
+                            seq.append(int(cp.group(1), 16))
+                        cp_list.append(seq)
+                if len(cp_list) == 0:
+                    print(
+                        f"Failed to get code points: {emoji_codes}, {emoji_name}",
+                        file=sys.stderr,
+                    )
+                    continue
+
+                value_name = f'"{emoji_name}"'
+                value_code_text = f'"{emoji_codes}"'
+                value_block = f'"{emoji_type}"'
+                for code in cp_list:
+                    if type(code) is int:
+                        value_code_text = f'"{code:X}"'
+                        char = chr(code)
+                        value_char = f'"{char}"'
+                    else:
+                        char = ""
+                        for c in code:
+                            char = char + chr(c)
+                        value_char = f'"{char}"'
+
+                    try:
+                        id = table_char_autoincrement_id.next()
+                        dml = f"insert into char(id, name, codetext, char, block) values({id}, {value_name}, {value_code_text}, {value_char}, {value_block})"
+                        cur.execute(dml)
+                        if type(code) is int:
+                            dml_seq = f"insert into codepoint(char, seq, code) values({id}, 1, {code})"
+                            cur.execute(dml_seq)
+                        else:
+                            for i, c in enumerate(code):
+                                dml_seq = f"insert into codepoint(char, seq, code) values({id}, {i}, {c})"
+                                cur.execute(dml_seq)
+                        count = count + 1
+                    except sqlite3.IntegrityError:
+                        print(
+                            f"Already registered: {value_code_text} {value_name}",
+                            file=sys.stderr,
+                        )
+
+            conn.commit()
+            print(f"Stored {count} emoji characters", file=sys.stderr)
+
+
+def create_database():
+    """Create Unicode database"""
+    Database().create()
+    store_ucd(
+        download_ucd("https://www.unicode.org/Public/15.0.0/ucdxml/ucd.all.flat.zip")
+    )
+    store_emoji(
+        download_emoji("https://www.unicode.org/Public/emoji/15.0/emoji-sequences.txt")
+    )
+    store_emoji(
+        download_emoji(
+            "https://www.unicode.org/Public/emoji/15.0/emoji-zwj-sequences.txt"
+        )
+    )
+    return 0
+
+
+def delete_database():
+    """Delete Unicode database"""
+    Database().delete()
+    return 0
+
+
+def database_info():
+    """Show database information"""
+    print(Database().get_path())
+    return 0

unicode_tools/db.py

@@ -0,0 +1,141 @@
+import os
+import sqlite3
+import sys
+from pathlib import Path
+
+
+def get_data_dir():
+    """データディレクトリのパスを決定する"""
+    # 環境変数でオーバーライド可能
+    if "UNICODE_DB_PATH" in os.environ:
+        return Path(os.environ["UNICODE_DB_PATH"]).parent
+
+    if os.getuid() == 0:  # rootユーザー
+        # /procを使って親プロセスをチェック（Linuxのみ）
+        try:
+            with open("/proc/self/stat") as f:
+                stats = f.read().split()
+                parent_pid = int(stats[3])  # 4番目が親プロセスID
+
+            with open(f"/proc/{parent_pid}/comm") as f:
+                parent_name = f.read().strip()
+
+            if parent_name in ["systemd", "cron"]:
+                # システムサービスとして実行されている
+                return Path("/var/lib/unicode-tools")
+        except (FileNotFoundError, ValueError, IndexError):
+            # /procが読めない場合やLinux以外の場合
+            pass
+
+        # rootの個人利用
+        return Path("/root/.local/share/unicode-tools")
+    else:
+        # 通常ユーザー
+        return Path.home() / ".local/share/unicode-tools"
+
+
+# データベースパスの設定
+data_dir = get_data_dir()
+unicode_sqlite3_database_path = str(data_dir / "unicode.db")
+unicode_sqlite3_database_dir = str(data_dir)
+
+# ディレクトリが存在しない場合は作成（ただし警告を出す）
+if not os.path.exists(unicode_sqlite3_database_dir):
+    try:
+        os.makedirs(unicode_sqlite3_database_dir)
+        print(f"Created directory: {unicode_sqlite3_database_dir}", file=sys.stderr)
+    except PermissionError:
+        print(
+            f"Permission denied: Cannot create directory {unicode_sqlite3_database_dir}",
+            file=sys.stderr,
+        )
+        print(
+            "Please create the directory manually or set UNICODE_DB_PATH environment variable",
+            file=sys.stderr,
+        )
+        sys.exit(1)
+
+
+class Database:
+    def create(self):
+        def execute(conn, name, dml):
+            with Cursor(conn) as cur:
+                try:
+                    cur.execute(dml)
+                    conn.commit()
+                    print(f"Created table: {name}")
+                except Exception as e:
+                    t = type(e)
+                    s = str(e)
+                    if t == sqlite3.OperationalError and s.endswith(" already exists"):
+                        print(f"Table already exists: {name}", file=sys.stderr)
+                    else:
+                        print(f"Failed to create table: {name}", file=sys.stderr)
+                        print(f"{type(e).__name__}: {str(e)}", file=sys.stderr)
+
+        with Connection() as conn:
+            execute(
+                conn,
+                "char",
+                "create table char(id integer primary key, name text, detail text, codetext text, char text, block text)",
+            )
+            execute(
+                conn,
+                "codepoint",
+                "create table codepoint(char integer, seq integer, code integer, primary key(char, seq))",
+            )
+            execute(
+                conn, "char_index", "create unique index char_index on char(codetext)"
+            )
+
+    def delete(self):
+        if not os.path.exists(unicode_sqlite3_database_path):
+            print(f"No database file: {unicode_sqlite3_database_path}", file=sys.stderr)
+        else:
+            os.remove(unicode_sqlite3_database_path)
+            print(
+                f"Deleted database file: {unicode_sqlite3_database_path}",
+                file=sys.stderr,
+            )
+
+    def get_path(self):
+        return unicode_sqlite3_database_path
+
+
+class Connection:
+    def __init__(self):
+        self.conn = None
+
+    def __enter__(self):
+        self.conn = sqlite3.connect(unicode_sqlite3_database_path)
+        return self.conn
+
+    def __exit__(self, *args):
+        if self.conn:
+            self.conn.close()
+
+
+class Cursor:
+    def __init__(self, conn):
+        self.conn = conn
+        self.cur = None
+
+    def __enter__(self):
+        self.cur = self.conn.cursor()
+        return self.cur
+
+    def __exit__(self, *args):
+        if self.cur:
+            self.cur.close()
+            self.cur = None
+
+
+class AutoID:
+    def init(self):
+        self.value = 1
+        return self
+
+    def next(self):
+        value = self.value
+        self.value = self.value + 1
+        return value

unicode_tools/normalize.py

@@ -0,0 +1,112 @@
+#!/usr/bin/env python3
+
+import sys
+import unicodedata
+
+
+def convert_to_halfwidth(text):
+    """Convert fullwidth characters to halfwidth"""
+    return "".join(
+        chr(ord(char) - 0xFEE0)
+        if 0xFF01 <= ord(char) <= 0xFF5E
+        else " "
+        if char == "　"  # Fullwidth space to regular space
+        else char
+        for char in text
+    )
+
+
+def get_binary_representation(text):
+    """Get binary representation of text as hex string with colon separators"""
+    return ":".join(f"{b:02x}" for b in text.encode("utf-8"))
+
+
+def get_unicode_representation(text):
+    """Get Unicode code point representation with colon separators"""
+    return ":".join(f"U+{ord(c):04X}" for c in text)
+
+
+def escape_control_chars(text):
+    """Replace control characters and space with dots for display"""
+    return "".join("." if ord(char) <= 32 else char for char in text)
+
+
+def format_detailed_output(text, form, delimiter=" "):
+    """Format text with detailed information: result form binary unicode"""
+    # Escape control characters in display text
+    display_text = escape_control_chars(text)
+    binary = get_binary_representation(text)
+    unicode_repr = get_unicode_representation(text)
+    return delimiter.join([display_text, form, binary, unicode_repr])
+
+
+def normalize_text(text, form="NFC", halfwidth=False):
+    """Normalize text using specified Unicode normalization form and halfwidth conversion"""
+    # Apply Unicode normalization
+    normalized = (
+        unicodedata.normalize(form.upper(), text)
+        if form.upper() in ["NFC", "NFD", "NFKC", "NFKD"]
+        else text
+    )
+
+    # Apply halfwidth conversion if specified
+    return convert_to_halfwidth(normalized) if halfwidth else normalized
+
+
+def analyze_normalization(content, delimiter=" "):
+    """Analyze text and show all normalization forms in consistent format"""
+    forms = ["Original", "NFC", "NFD", "NFKC", "NFKD"]
+
+    for form in forms:
+        if form == "Original":
+            result_text = content
+        else:
+            result_text = unicodedata.normalize(form, content)
+
+        print(format_detailed_output(result_text, form, delimiter))
+
+
+def normalize_command(
+    form="NFC",
+    halfwidth=False,
+    compare=False,
+    detailed=False,
+    delimiter=" ",
+    input_file=None,
+):
+    """Main normalize command function"""
+    try:
+        # Read entire content as single string
+        if input_file and input_file != "-":
+            with open(input_file, encoding="utf-8") as f:
+                content = f.read().rstrip("\n")
+        else:
+            content = sys.stdin.read().rstrip("\n")
+
+        # Process content
+        if compare:
+            analyze_normalization(content, delimiter)
+        else:
+            result = normalize_text(content, form=form, halfwidth=halfwidth)
+            if detailed:
+                # Determine actual form used
+                actual_form = form.upper()
+                if halfwidth:
+                    actual_form += "+HALFWIDTH"
+                print(format_detailed_output(result, actual_form, delimiter))
+            else:
+                print(
+                    result, end=""
+                )  # Don't add extra newline since content may already have it
+
+    except FileNotFoundError:
+        print(f"Error: File '{input_file}' not found", file=sys.stderr)
+        return 1
+    except UnicodeDecodeError:
+        print("Error: Unable to decode file as UTF-8", file=sys.stderr)
+        return 1
+    except BrokenPipeError:
+        # Handle pipe operations gracefully
+        pass
+
+    return 0

unicode_tools/search.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+
+import re
+
+from .db import Connection, Cursor
+
+
+def get_code_range(fragment):
+    if "-" in fragment:
+        m = re.match("([0-9A-Fa-f]+)-([0-9A-Fa-f]+)", fragment)
+        min = int(m.group(1), 16)
+        max = int(m.group(2), 16)
+        r = (min, max)
+    else:
+        m = re.match("[0-9A-Fa-f]+", fragment)
+        r = int(fragment, 16)
+    return r
+
+
+def search(fragment, by, delimiter, strict=False, first=False, format=None):
+    if format is not None:
+        format = format.upper()
+
+    with Connection() as conn:
+        char_list = []
+        head = "select char.id, char.codetext, char.name, char.char from char"
+        head_detail = "select char.id, char.codetext, char.detail, char.char from char"
+        if by == "code":
+            code_range = get_code_range(fragment)
+            if type(code_range) is tuple:
+                cond = (
+                    f"where cp.code >= {code_range[0]} and cp.code <= {code_range[1]}"
+                )
+            else:
+                cond = f"where cp.code = {code_range}"
+            dml = " ".join(
+                [
+                    head,
+                    "inner join codepoint as cp on char.id = cp.char",
+                    cond,
+                    "order by char.char",
+                ]
+            )
+        elif by == "char":
+            cond = f'where char.char = "{fragment}"'
+            dml = " ".join([head, cond])
+        else:
+            if strict:
+                by = f"upper({by})"
+                matched = f'= "{fragment.upper()}"'
+            else:
+                matched = f'like "%{fragment}%"'
+            if by == "detail":
+                dml = " ".join(
+                    [head_detail, "where", by, matched, "order by char.char"]
+                )
+            else:
+                dml = " ".join([head, "where", by, matched, "order by char.char"])
+
+        with Cursor(conn) as cur:
+            cur.execute(dml)
+            char_list = cur.fetchall()
+
+        if first == True:
+            char_list = char_list[0:1]
+
+        for id, codetext, name, char in char_list:
+            if not char:
+                char = str(char)
+
+            if format == "SIMPLE":
+                print(char, end="")
+            else:
+                if format == "UTF8":
+                    codetext = " ".join(
+                        f"{u:X}"
+                        for u in [
+                            int.from_bytes(chr(int(c, 16)).encode(), "big")
+                            for c in codetext.split(" ")
+                        ]
+                    )
+
+                print(delimiter.join([char, codetext, name]))

unicode_tools/uchr.py

@@ -0,0 +1,242 @@
+#!/usr/bin/env python3
+
+import argparse
+import io
+import sys
+
+
+def wrap_io():
+    """Wrap stdin/stdout/stderr with UTF-8 encoding"""
+    sys.stdin = io.TextIOWrapper(sys.stdin.buffer, encoding="utf-8")
+    sys.stdout = io.TextIOWrapper(
+        sys.stdout.buffer, encoding="utf-8", line_buffering=True
+    )
+    sys.stderr = io.TextIOWrapper(
+        sys.stderr.buffer, encoding="utf-8", line_buffering=True
+    )
+
+
+def search_command(args):
+    """Handle uchr search subcommand"""
+    from .search import search
+
+    # Convert args to match search.search signature
+    by = args.by if args.by else "name"
+
+    if (by == "code" or by == "char") and args.strict:
+        print(f"warning: Ignore --strict in {by} search")
+
+    search(
+        args.expression,
+        by,
+        args.delimiter,
+        strict=args.strict,
+        first=args.first,
+        format=args.format,
+    )
+
+
+def normalize_command(args):
+    """Handle uchr normalize subcommand"""
+    from .normalize import normalize_command as normalize_func
+
+    return normalize_func(
+        form=args.form,
+        halfwidth=args.halfwidth,
+        compare=args.compare,
+        detailed=args.detail,
+        delimiter=args.delimiter,
+        input_file=args.input_file,
+    )
+
+
+def db_create_command(args):
+    """Handle uchr db create subcommand"""
+    from .database import create_database
+
+    return create_database()
+
+
+def db_delete_command(args):
+    """Handle uchr db delete subcommand"""
+    from .database import delete_database
+
+    return delete_database()
+
+
+def db_info_command(args):
+    """Handle uchr db info subcommand"""
+    from .database import database_info
+
+    return database_info()
+
+
+def create_parser():
+    """Create the main argument parser with subcommands"""
+    parser = argparse.ArgumentParser(
+        prog="uchr",
+        description="Unicode character tools",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+  uchr search ghost                    # Search for characters named 'ghost'
+  uchr search -c 1F47A-1F480          # Search by code range
+  uchr search -x 👻                   # Search by character
+  uchr search -b "Emoticons"          # Search by Unicode block
+  uchr normalize                      # Normalize text from stdin (NFC)
+  uchr normalize --form nfd           # Normalize to NFD form
+  uchr normalize --halfwidth          # Convert fullwidth to halfwidth
+  uchr normalize --detail             # Show result form binary unicode
+  uchr normalize --compare            # Show all normalization forms
+  uchr db create                      # Create Unicode database
+  uchr db info                        # Show database location
+        """,
+    )
+
+    subparsers = parser.add_subparsers(dest="command", help="Available commands")
+
+    # Search subcommand
+    search_parser = subparsers.add_parser("search", help="Search Unicode characters")
+    search_parser.add_argument(
+        "expression", metavar="EXPR", help="Expression to search"
+    )
+
+    search_by_group = search_parser.add_mutually_exclusive_group()
+    search_by_group.add_argument(
+        "-b",
+        "--block",
+        action="store_const",
+        dest="by",
+        const="block",
+        help="Search by block name",
+    )
+    search_by_group.add_argument(
+        "-c",
+        "--code",
+        action="store_const",
+        dest="by",
+        const="code",
+        help="Search by code point or range",
+    )
+    search_by_group.add_argument(
+        "-x",
+        "--char",
+        action="store_const",
+        dest="by",
+        const="char",
+        help="Search by character",
+    )
+    search_by_group.add_argument(
+        "-d",
+        "--detail",
+        action="store_const",
+        dest="by",
+        const="detail",
+        help="Search by character details",
+    )
+
+    search_parser.add_argument(
+        "-s",
+        "--strict",
+        action="store_true",
+        help="Match name strictly (case insensitive)",
+    )
+    search_parser.add_argument(
+        "-1", "--first", action="store_true", help="Show first result only"
+    )
+    search_parser.add_argument(
+        "-f", "--format", choices=["utf8", "simple"], default=None, help="Output format"
+    )
+    search_parser.add_argument(
+        "-D", "--delimiter", default=" ", help="Output delimiter (default: space)"
+    )
+    search_parser.set_defaults(func=search_command)
+
+    # Normalize subcommand
+    normalize_parser = subparsers.add_parser(
+        "normalize", help="Normalize and convert Unicode text"
+    )
+    normalize_parser.add_argument(
+        "input_file", nargs="?", default=None, help="Input file (default: stdin)"
+    )
+    normalize_parser.add_argument(
+        "--form",
+        choices=["nfc", "nfd", "nfkc", "nfkd"],
+        default="nfc",
+        help="Unicode normalization form (default: nfc)",
+    )
+    normalize_parser.add_argument(
+        "--halfwidth",
+        action="store_true",
+        help="Convert fullwidth characters to halfwidth",
+    )
+    normalize_parser.add_argument(
+        "--compare",
+        action="store_true",
+        help="Show comparison of all normalization forms",
+    )
+    normalize_parser.add_argument(
+        "--detail",
+        action="store_true",
+        help="Show detailed output: result form binary unicode",
+    )
+    normalize_parser.add_argument(
+        "--delimiter",
+        default=" ",
+        help="Delimiter for detailed output (default: space)",
+    )
+    normalize_parser.set_defaults(func=normalize_command)
+
+    # Database subcommand
+    db_parser = subparsers.add_parser("db", help="Database management")
+    db_subparsers = db_parser.add_subparsers(
+        dest="db_command", help="Database operations"
+    )
+
+    # db create
+    db_create_parser = db_subparsers.add_parser(
+        "create", help="Create Unicode database"
+    )
+    db_create_parser.set_defaults(func=db_create_command)
+
+    # db delete
+    db_delete_parser = db_subparsers.add_parser(
+        "delete", help="Delete Unicode database"
+    )
+    db_delete_parser.set_defaults(func=db_delete_command)
+
+    # db info
+    db_info_parser = db_subparsers.add_parser("info", help="Show database information")
+    db_info_parser.set_defaults(func=db_info_command)
+
+    return parser
+
+
+def main():
+    """Main entry point for uchr command"""
+    wrap_io()
+
+    parser = create_parser()
+    args = parser.parse_args()
+
+    if not hasattr(args, "func"):
+        parser.print_help()
+        return 1
+
+    try:
+        return args.func(args) or 0
+    except KeyboardInterrupt:
+        print("\nInterrupted", file=sys.stderr)
+        return 1
+    except Exception as e:
+        print(f"Error: {e}", file=sys.stderr)
+        return 1
+
+
+def uchr():
+    """Entry point for console_scripts"""
+    return main()
+
+
+if __name__ == "__main__":
+    sys.exit(main())