Wildcard

# Wildcard A high-performance .NET 10 toolkit for wildcard pattern matching, file system globbing, and content search. Zero allocations on the hot path, memory-mapped file scanning, and a CLI grep tool (`wcg`) that gets within 19% of ripgrep. Ships with an MCP server that gives AI coding agents fast, directory-restricted file search and replace. ## Requirements - .NET 10.0 or later ## Supported Syntax | Token | Description | |-------|-------------| | `*` | Matches any sequence of characters (including empty) | | `?` | Matches exactly one character | | `[abc]` | Matches any character in the set | | `[a-z]` | Matches any character in the range | | `[!x]` or `[^x]` | Matches any character NOT in the set | | `\` | Escapes the next character (e.g. `\*` matches a literal `*`) | | `{a,b,c}` | Brace expansion — matches any of the comma-separated alternatives | ### Glob-only Syntax | Token | Description | |-------|-------------| | `**` | Matches zero or more directory levels (recursive) | ## Usage ```csharp // One-shot match bool matches = WildcardPattern.IsMatch("*.cs", "program.cs"); // true // Pre-compiled pattern (reuse across many inputs) var pattern = WildcardPattern.Compile("file?.log"); pattern.IsMatch("file1.log"); // true pattern.IsMatch("fileAB.log"); // false // Case-insensitive matching WildcardPattern.IsMatch("HELLO", "hello", ignoreCase: true); // true // Brace alternation — match any of the comma-separated alternatives WildcardPattern.IsMatch("*.{cs,fs,vb}", "program.cs"); // true WildcardPattern.IsMatch("{error,warn}*", "error: timeout"); // true WildcardPattern.IsMatch("{get,set}_{name,value}", "get_name");// true // TryMatch — extract what each * captured var p = WildcardPattern.Compile("\\[*\\] * - *"); if (p.TryMatch("[2024-03-15] ERROR - timeout", out var captures)) { // captures: ["2024-03-15", "ERROR", "timeout"] } // LINQ extensions string[] files = ["app.cs", "readme.md", "test.cs", "data.csv"]; var csFiles = files.WhereMatch("*.cs").ToList(); // ["app.cs", "test.cs"] bool hasCsv = files.AnyMatch(WildcardPattern.Compile("*.csv")); // true string? first = files.FirstMatch(WildcardPattern.Compile("*.md")); // "readme.md" // Convert to Regex for interop Regex regex = WildcardPattern.Compile("*.csv").ToRegex(); // regex.ToString() == "^.*\\.csv$" // Convert to a Func<string, bool> predicate PatternPredicate predicate = WildcardPattern.ToPredicate("*.cs"); predicate("app.cs"); // true predicate("readme.md"); // false // Bulk filtering var pattern = WildcardPattern.Compile("*.cs"); List<string> csharpFiles = WildcardSearch.FilterLines(pattern, files); // Parallel bulk filtering for large datasets string[] results = WildcardSearch.FilterBulk(pattern, largeArray, parallel: true); ``` ### File Content Scanning `FilePathMatcher` scans files on disk for lines matching wildcard patterns using memory-mapped I/O. ```csharp // Single include pattern var matcher = FilePathMatcher.Create("*ERROR*"); List<FilePathMatcher.LineMatch> matches = matcher.Scan("app.log", "server.log"); foreach (var match in matches) Console.WriteLine($"{match.FilePath}:{match.LineNumber}: {match.Line}"); // Multiple include patterns — OR logic (match lines containing ERROR or WARN) var matcher = FilePathMatcher.Create( include: ["*ERROR*", "*WARN*"] ); var matches = matcher.Scan("app.log"); // Include/exclude patterns — match lines containing ERROR but not DEBUG var matcher = FilePathMatcher.Create( include: ["*ERROR*"], exclude: ["*DEBUG*"] ); var matches = matcher.Scan("app.log"); // Quick boolean check — does a file contain any match? var matcher = FilePathMatcher.Create("*timeout*"); bool hasTimeout = matcher.ContainsMatch("app.log"); // Async streaming — results arrive as they are found await foreach (var match in matcher.ScanAsync(filePaths)) Console.WriteLine(match.Line); // Context lines — like grep -C 3 (3 lines before and after each match) var matcher = FilePathMatcher.Create("*ERROR*"); List<FilePathMatcher.ContextLine> results = matcher.ScanWithContext( beforeContext: 3, afterContext: 3, "app.log"); foreach (var line in results) { var sep = line.IsMatch ? ":" : "-"; Console.WriteLine($"{line.LineNumber}{sep} {line.Line}"); } ``` ### File System Globbing `Glob` matches file paths on disk with support for `*`, `?`, `[abc]`, `**` (recursive directory matching), and `{a,b,c}` (brace expansion). ```csharp // Find all .cs files recursively var files = Glob.Match("src/**/*.cs").ToList(); // Brace expansion — match multiple extensions in one pattern var webFiles = Glob.Match("**/*.{razor,cs,css}").ToList(); // Single directory level var logs = Glob.Match("/var/log/*.log").ToList(); // Respect .gitignore — skips bin/, obj/, node_modules/, .git/ etc. var tracked = Glob.Match("**/*.cs", options: new GlobOptions { RespectGitignore = true }).ToList(); // Follow symbolic links (off by default, matching ripgrep behavior) var withSymlinks = Glob.Match("**/*.cs", options: new GlobOptions { FollowSymlinks = true }).ToList(); // Pre-parsed glob for reuse var glob = Glob.Parse("**/*.json"); foreach (var file in glob.EnumerateMatches("/my/project")) Console.WriteLine(file); // Path matching without touching the filesystem Glob.IsMatch("src/**/*.cs", "src/Models/User.cs"); // true ``` ### Find and Replace `FileReplacer` performs find-and-replace across files with dry-run preview, atomic writes, and encoding preservation. ```csharp // Preview replacements (no files modified) var results = FileReplacer.Preview(filePaths, "oldMethod", "newMethod"); foreach (var file in results) foreach (var r in file.Replacements) Console.WriteLine($"{file.FilePath}:{r.LineNumber}: {r.OriginalLine} → {r.ReplacedLine}"); // Apply replacements (atomic write per file) FileReplacer.Apply(filePaths, "ERROR", "WARNING", ignoreCase: true); // Capture-group replacement — wildcards in find, $1/$2 in replace var results = FileReplacer.Preview(filePaths, "*console.log(*)*", "$1logger.info($2)$3"); // Multi-line literal find — always matched literally (wildcards can't span lines); // line endings in find/replace are normalized to each file's own style FileReplacer.Apply(filePaths, " DoOld();\n DoOld2();", " DoNew();"); ``` Safety: skips binary files, read-only files, and files over 10MB. Preserves encoding (BOM) and line endings (`\r\n`/`\n`). Writes atomically via temp file + rename. If a file fails (permissions, locked), the error is reported and the remaining files continue processing. ### CLI Tool — `wcg` A command-line grep tool built on top of the library. Respects `.gitignore` by default, streams results as they're found. #### Install **Option A — .NET tool** (cross-platform, requires .NET 10 runtime): ```bash dotnet tool install -g wcg dotnet tool update -g wcg # to update ``` **Option B — Native binary** (single file, no dependencies, ~4MB): Download the latest binary for your platform from [GitHub Releases](https://github.com/Jabe/Wildcard/releases): | Platform | Binary | |----------|--------| | macOS (Apple Silicon) | `wcg-osx-arm64` | | Linux (x64) | `wcg-linux-x64` | | Windows (x64) | `wcg-win-x64.exe` | ```bash # macOS / Linux chmod +x wcg-osx-arm64 sudo mv wcg-osx-arm64 /usr/local/bin/wcg ``` #### Usage ``` wcg <glob> [<pattern>...] [options] Arguments: <glob> File glob pattern (e.g. "src/**/*.cs") <pattern> Content search pattern(s) — multiple patterns are OR'd (e.g. ERROR WARN). Plain words match as substrings; use wildcards for prefix/suffix/full patterns (e.g. "ERROR*", "*.log"). Options: -x, --exclude <pattern> Exclude lines matching pattern (repeatable) -X, --exclude-path <glob> Exclude files matching glob (repeatable) -i, --ignore-case Case-insensitive content matching -l, --files-with-matches Only print file paths that contain matches --no-ignore Don't respect .gitignore files -L, --follow Follow symbolic links -w, --watch Watch for changes after initial scan -A, --after-context <N> Show N lines after each match -B, --before-context <N> Show N lines before each match -C, --context <N> Show N lines before and after each match -c, --count Show count of matching lines per file --tree Render file list as an indented ASCII tree --depth <N> Max directory depth for tree output (default: 5) --all AND mode — file must contain ALL patterns --max-files <N> Max files to include in output --max-matches <N> Max matches to show per file -r, --replace <text> Replace matched content with this string (dry-run preview) --write Write replacements to files (requires --replace) ``` Examples: ```bash wcg "src/**/*.cs" # List matching files wcg "src/**/*.cs" --tree # List matching files as an ASCII tree wcg "src/**/*.cs" --tree --depth 2 # Tree capped at 2 levels deep wcg "**/*.log" ERROR # Search for lines containing ERROR wcg "**/*.log" ERROR WARN # OR mode — containing ERROR or WARN wcg "**/*.log" ERROR WARN --all # AND mode — file must contain both wcg "**/*.cs" TODO -x DONE # Search TODO, exclude DONE wcg "**/*.cs" TODO FIXME -x DONE # Search TODO or FIXME, exclude DONE wcg "**/*.cs" TODO -i # Case-insensitive search wcg "**/*.log" ERROR --watch # Watch for new ERROR lines wcg "**/*" class -X "*test*" # Search, skip test paths wcg "**/*.cs" --no-ignore # Include .gitignore'd files wcg "**/*.cs" -L # Follow symbolic links wcg "**/*.cs" TODO -C 3 # Show 3 lines of context around matches wcg "**/*.log" ERROR -B 2 -A 5 # 2 lines before, 5 lines after each match # Plain words are auto-wrapped as *word* (substring match). # Use explicit wildcards for prefix/suffix/pattern matching: wcg "**/*.cs" "using*" # Lines starting with "using" wcg "**/*.log" "*.json" # Lines ending with ".json" wcg "**/*.log" "*ERROR*timeout*" # Multi-segment wildcard # ? matches exactly one character: wcg "**/*.log" "ERR?R" # Matches ERROR, ERRIR, ERR0R, … wcg "**/*.log" "v?.?.?" # Matches v1.2.3, v2.0.1, … # Character classes: wcg "**/*.log" "HTTP [45]??" # HTTP 4xx or 5xx — [45] + two ?? digits wcg "**/*.log" "[EIWD]*" # Lines starting with E, I, W or D (ERROR/INFO/WARN/DEBUG) wcg "**/*.log" "[!D]*" # Lines not starting with D (excludes DEBUG) # Brace alternation in content patterns: wcg "**/*.log" "{ERROR,WARN}*" # Lines starting with ERROR or WARN wcg "**/*.cs" "{TODO,FIXME,HACK}*" # Lines starting with any of those markers # Output caps: wcg "**/*.log" ERROR --max-files 10 # Stop after 10 files wcg "**/*.log" ERROR --max-matches 5 # Show at most 5 matches per file # Count mode: wcg "**/*.cs" TODO -c # Per-file match counts + summary wcg "**/*.cs" TODO -c -l # Summary only (total matches and files) wcg "**/*.cs" -c # Count matching files (no content pattern) # Find and replace (dry-run preview by default): wcg "**/*.cs" oldMethod --replace newMethod # Preview replacements wcg "**/*.cs" oldMethod --replace newMethod --write # Apply changes wcg "**/*.cs" ERROR --replace WARNING -i # Case-insensitive replace # Capture-group replacement (wildcards in find, $1/$2 in replace): wcg "**/*.cs" "*console.log(*)*" -r '$1logger.info($2)$3' # Refactor method calls ``` ### MCP Server — `wildcard-mcp` An [MCP](https://modelcontextprotocol.io) (Model Context Protocol) server that exposes Wildcard's capabilities as tools for AI coding assistants (Copilot, Claude, Cursor, etc.). #### Tools | Tool | Description | |------|-------------| | `wildcard_glob` | Find files by glob pattern — respects `.gitignore`, supports count mode and tree output | | `wildcard_grep` | Search file contents with context lines, count mode, AND/OR mode, output caps, and parallel memory-mapped I/O. Also acts as a cross-platform file reader (cat/head) | | `wildcard_replace` | Find-and-replace across files — dry-run preview by default, atomic writes, capture groups, multi-line literal find | | `wildcard_peek` | Batch file reader — read multiple files in one call with optional line ranges and a character budget | | `wildcard_watch` | Watch for file changes matching a glob pattern for a bounded duration | #### Install ```bash dotnet tool install -g wildcard-mcp ``` #### VS Code / Copilot Add to `.vscode/mcp.json`: ```json { "servers": { "wildcard": { "type": "stdio", "command": "wildcard-mcp", "args": ["--live"] } } } ``` #### Claude Code Add to `.mcp.json` in your project root: ```json { "mcpServers": { "wildcard": { "command": "wildcard-mcp", "args": ["--live"] } } } ``` The server communicates over stdio and auto-discovers all five tools. All operations are **restricted to the workspace roots** declared by the MCP client — path traversal outside the allowed roots is rejected by a guard that resolves symlinks and validates every path before any file I/O. The server requires the client to support the MCP [roots](https://modelcontextprotocol.io/docs/concepts/roots) capability; multiple roots are supported and dynamic root changes are handled via `notifications/roots/list_changed`. Pass `--live` to enable live mode: the server builds an in-memory index of all files (one per root) at startup and keeps it current via filesystem watcher. Glob queries become an in-memory filter instead of a disk walk, which dramatically speeds up repeated searches on large codebases. Pass `--summary` to prepend a one-line echo of the call arguments to each tool result, e.g. `[pattern="**/*.cs", ignore_case=true]`. Only arguments the client actually sent appear (defaults stay out), so you can audit what the LLM passed without opening the raw request. Off by default. ## Benchmarks Measured on Apple M4 Pro, .NET 10.0.7, Arm64 RyuJIT AdvSIMD. Zero allocations for all single-match operations. ### Pattern Matching — Wildcard vs Compiled Regex vs FileSystemName | Pattern | Input | Wildcard | Regex | FSName | Speedup vs Regex | |---------|-------|----------|-------|--------|-----------------| | `*.csv` | short (15 chars) | 1.3 ns | 13.0 ns | 4.5 ns | **10x** | | `*.csv` | long (74 chars) | 1.3 ns | 15.0 ns | 4.5 ns | **12x** | | `*.csv` | no match | 1.3 ns | 10.2 ns | 4.2 ns | **8x** | | `report*.csv` | short | 2.4 ns | 13.3 ns | 100.7 ns | **5x** | | `report_????.csv` | short | 5.3 ns | 9.0 ns | 47.3 ns | **2x** | | `[rs]*.*` | short | 7.6 ns | 13.3 ns | — | **2x** | | `*report*2024*` | short | 12.7 ns | 22.0 ns | 236.7 ns | **2x** | | `*report*2024*` | long | 14.9 ns | 25.4 ns | 1,199.7 ns | **2x** | ### Real-World Patterns — Wildcard vs Compiled Regex | Pattern | Scenario | Wildcard | Regex | Speedup | |---------|----------|----------|-------|---------| | `v2.*` | version prefix | 0.8 ns | 8.0 ns | **10x** | | `[[]2024-03-15*` | log date prefix | 1.0 ns | 8.1 ns | **8x** | | `*@gmail.com` | email domain | 1.5 ns | 10.0 ns | **7x** | | `[AEIOU]*` (CI) | starts with vowel | 2.4 ns | 6.2 ns | **3x** | | `??? *` | 3-char first name | 3.0 ns | 6.2 ns | **2x** | | `J* *Smith` (CI) | no match | 4.5 ns | 6.2 ns | **1.4x** | | `*ERROR*timeout*` | log search (no match) | 6.9 ns | 8.2 ns | **1.2x** | | `*@*.acme-corp.com` | corporate email | 7.7 ns | 15.6 ns | **2x** | | `SKU-*-BLUE-*` | product code | 9.3 ns | 10.1 ns | **1.1x** | | `J* *Smith` (CI) | match | 10.3 ns | 14.7 ns | **1.4x** | | `*ERROR*timeout*` | log search (match) | 15.3 ns | 20.0 ns | **1.3x** | ### Bulk Filtering — 10,000 Items | Method | Mean | Allocated | |--------|------|-----------| | Wildcard FilterLines | 34 µs | 33 KB | | Wildcard FilterBulk (parallel) | 79 µs | 174 KB | | FSName LINQ filter | 67 µs | 10 KB | | Regex LINQ filter | 200 µs | 11 KB | ### File Content Scanning — `FilePathMatcher` Pattern `*ERROR*` across 4 log files (~12.5% matching lines). Compared against `File.ReadAllLines` + `FilterLines` baseline. | File size | Baseline (ReadAllLines) | FilePathMatcher | Ratio | Alloc Ratio | |-----------|------------------------|-----------------|-------|-------------| | small (1K lines) | 271 µs | 73 µs | 0.27 | 0.15 | | medium (100K lines) | 52,220 µs | 7,626 µs | 0.15 | 0.15 | | large (1M lines) | 559,739 µs | 108,121 µs | 0.19 | 0.15 | ### CLI — `wcg` vs `find+grep`, `ripgrep` Real-world content search on `~/Code` (~130k files across multiple git repos). Apple M4 Pro, .NET 10.0. | Tool | Wall time | CPU% | |------|-----------|------| | `find+grep` | 17.9s | 39% | | `wcg` (dotnet tool) | 1.66s | 478% | | `wcg` (native AOT) | 1.50s | 447% | | `rg` (ripgrep) | **1.26s** | 316% | `wcg` is within **19% of ripgrep** for content search. `.gitignore` filtering (on by default) prunes `bin/`, `obj/`, `node_modules/` etc. during traversal. Work-stealing parallel glob overlaps directory enumeration with I/O-bound content scanning. The native AOT binary adds ~10% wall-clock improvement and 10ms cold startup. ## How It Works ### 1. Pattern Compilation `PatternCompiler.Compile` parses a pattern string into an array of `Segment` objects. Each segment is one of five types: - **Literal** — a fixed string to match exactly (e.g. `".cs"`) - **Star** — the `*` wildcard, matches any character sequence - **QuestionMark** — the `?` wildcard, matches exactly one character - **QuestionRun** — consecutive `?` characters collapsed into a single segment with a count - **CharClass** — a character set like `[a-z]`, optionally negated, using `SearchValues<char>` for SIMD-accelerated lookups Consecutive `*` characters are collapsed into a single Star segment during compilation. Single-character, non-negated character classes (e.g. `[[]`) are promoted to Literal segments and merged with adjacent literals. **Brace expansion** — patterns containing `{a,b,c}` are expanded by `BraceExpander` into multiple alternative patterns before compilation. For example, `*.{cs,fs}` becomes `["*.cs", "*.fs"]`. Each alternative is compiled independently and stored as a `WildcardPattern[]`. Matching succeeds if any alternative matches. Patterns without braces skip this step entirely (zero overhead). ### 2. Pattern Shape Specialization At compile time, common pattern shapes are detected and dispatched to optimized fast-paths that bypass the general matching engine entirely: | Shape | Example | Fast-path | |-------|---------|-----------| | `PureLiteral` | `hello` | `SequenceEqual` | | `StarSuffix` | `*.csv` | `EndsWith` | | `PrefixStar` | `v2.*` | `StartsWith` | | `PrefixStarSuffix` | `report*.csv` | `StartsWith` + `EndsWith` | | `StarContainsStar` | `*ERROR*` | `IndexOf` | All other patterns fall through to the general backtracking engine. ### 3. Matching Engine `MatchCore` walks the segment array and the input string simultaneously using a backtracking algorithm: - Two pointers track the current position in the segments and the input. - When a `*` segment is encountered, the engine records its position as a backtrack point and advances to the next segment. - If a subsequent segment fails to match, the engine backtracks to the last `*` position and tries consuming one more character from the input. - **IndexOf acceleration** — when a `*` is followed by a literal, the engine uses `Span.IndexOf` to jump directly to the next occurrence instead of scanning character-by-character. - **EndsWith fast-path** — when a `*` is followed by the final literal segment, the engine checks `EndsWith` instead of scanning. This approach avoids the exponential worst-case that naive recursive implementations can hit. ### 4. Performance Techniques - **Zero-copy matching** — uses `ReadOnlySpan<char>` to avoid string allocations during matching. - **Aggressive inlining** — hot-path methods like `MatchLiteral` and `CharsEqual` use `[MethodImpl(MethodImplOptions.AggressiveInlining)]`. - **SIMD-accelerated character classes** — `SearchValues<char>` provides hardware-accelerated membership testing. For case-insensitive patterns, both upper and lower case variants are expanded at compile time so the SIMD path works unconditionally. - **`ref readonly` struct access** — segments are accessed by reference in the hot loop to avoid copying the struct on each iteration. - **Parallel bulk operations** — `WildcardSearch.FilterBulk` processes arrays of 1024+ items in parallel using PLINQ with order preservation. ### 5. File Content Scanning `FilePathMatcher` scans files on disk using memory-mapped I/O and parallel processing: - **Adaptive file I/O** — small files (≤64KB) use pooled buffered reads; larger files use memory-mapped I/O. Files over 2GB are processed in 1GB overlapping sections. - **Byte-level pre-filtering** — for ASCII, case-sensitive patterns over UTF-8 data, pattern matching runs directly on raw bytes using SIMD-accelerated span operations (`IndexOf`, `StartsWith`, `EndsWith`). Lines that don't match skip UTF-8 decoding entirely. When multiple include patterns are given, each pattern gets its own byte-level filter; a line is skipped only when all filters reject it. - **Minimum length gate** — lines shorter than the pattern's minimum possible match length are rejected before any decoding or matching. - **Parallel multi-file scanning** — multiple files are scanned concurrently via `Parallel.For`, with results merged preserving file order. - **Async streaming** — `ScanAsync` uses a bounded channel to stream matches as they are found. ## Attribution This project was generated with the assistance of [Claude Opus 4.6](https://claude.ai) by [Anthropic](https://anthropic.com).