J.R.

System architecture

Responsibilities

• Two-stage architecture: stage one converts most of each script deterministically (preprocess → Lark parse → AST → rule engine → C# generation); stage two calls an LLM only for CALL :label subroutines the rules cannot express, keeping AI scope and risk minimal
• CMD grammar modelling: designed and kept extending a ~490-line Lark (LALR) grammar that copes with cmd.exe quirks — redirects anywhere, the ambiguity of < > & |, %VAR:old=new% substitution, many IF/ELSE bracket styles, labels containing Japanese
• Input-normalisation layer (8 passes): unifies mixed line endings, glued closing brackets, multi-line ELSE, ECHO metacharacters and full-width spaces without changing semantics, so the grammar stays small
• AST and rule engine: 27 custom AST node types; 33 built-in commands plus a 1,084-entry external-command mapping table (job-process and job-flow call modes); variable expansion handled separately for string, argument-line and integer-expression contexts
• LLM subroutine conversion with semantic-preservation checks: LabelMapper aligns the .cs TODO, the .cs.ast call node and the .cmd source; structural and business-semantic validators (ERRORLEVEL, cancel branches, error jumps and status flags must survive); on failure the reason is fed back for up to 7 retries, otherwise the TODO is kept for a human
• Cost and stability: JSON cache keyed by SHA-256 of the normalised label block, exponential back-off on 429/5xx, automatic fallback from Chat Completions to the Responses API, temperature=0 for reproducibility
• Quality loop: ~465 test cases over the grammar and transpiler layers; batch runs emit .cs.ERR files that are classified into four causes (CMD source issue / grammar gap / code defect / environment), with a shared list of common errors

Outcomes

• ✓ Found several latent defects in the legacy CMD during migration (e.g. a missing & that means a GOTO never actually runs; an unescaped < when ECHO writes SQL) and logged them as legacy-system issues
• ✓ Example: a 117-line data-linkage CMD became 156 lines of C#; the remaining 11 TODOs are all external commands missing from the mapping table and shrink as the table grows

Data flow

Legacy CMD → encoding detection → input normalisation → Lark parse → AST → rule engine → C# generation; TODO subroutines → three-way alignment → LLM generation → two-layer validation (≤7 retries) → .label.ai.cs

System architecture

Responsibilities

• Data-source migration: when the old staff directory was retired, moved the crawler to the new site — new page structure and Microsoft SSO login — while keeping the legacy role markers so downstream sync logic needed no change
• Selenium (headless Edge) logs in automatically, walks the org tree and looks up each person's unit and role; results are pushed one-way from the intranet PC to an Azure server via paramiko SFTP
• Sync batch: diffs old and new data, generates and runs SQL for account creation, deactivation, re-hire unlock and department/section/role sync; updates the reporting permission groups through the Redmine REST API, leaving allow-listed members untouched
• Three kinds of SendGrid alert e-mails; on an unexpected organisation change the job aborts and the SQL runs only after the owner confirms it (two-phase), so bad data never lands in production directly
• Centralised department mappings scattered in code into a database config table; handled the April reorganisation per the customer's policy and designed/ran test cases covering screens, the application flow and permission groups
• Diagnosed and fixed production issues: lock mis-detection from duplicate people, missing accounts for re-hires, page time-outs under network jitter, Edge driver mismatch
• Released to two production environments (intranet PC + Azure VM): backup, deploy, dry-run with DB writes disabled to verify the SQL, then go live; wrote the operations runbook (deployment, alert guide, troubleshooting, org-change SOP) and handed over
• Ran Fortify / FOSSology security and licence scans over the whole project, fixed high-severity findings such as JSON and SQL injection, and compiled the OSS licence list

Data flow

Staff directory → Selenium crawler (intranet PC) → CSV → SFTP → diff SQL / Redmine REST API (Azure VM) → accounts, organisation, permission groups

System architecture

Responsibilities

• Model validation and change of course: benchmarked Gemma-3 4B on COBOL comprehension; LoRA fine-tuning on Colab could not finish because of memory limits. Since the business dictionary changes per customer, moved to RAG, which needs no retraining
• Knowledge base: unified DB design manuals, glossaries and table/file/job-flow definitions into {key, value}, split semantically by file → record → field → group; embedded only the key (multilingual-e5-small) into ChromaDB, stripping OCCURS subscripts before lookup and building aliases and hierarchy paths
• COBOL parsing: generated a Python lexer/parser/visitor from Cobol85.g4; preprocessing for Shift_JIS→UTF-8, line-number removal and dialect directives such as $SET; turned the parse tree into a compact AST for the LLM; a variable extractor handling REDEFINES / OCCURS / COMP / level-88
• Call-hierarchy analysis: built a call graph from PERFORM, dropped EXT paragraphs that only serve as exits in RTN/EXT pairs, and used a topological sort to fix a bottom-up analysis order
• MCP integration: exposed vector search through FastMCP as four tools — lookup_field, lookup_record, batch_lookup_fields, get_subroutine_summary — so the model fetches only the variables it uses instead of hundreds of definitions
• Generation and prompt tuning: per subroutine, extract input/output/work variables and callees, look definitions up via MCP, pass lower-level summaries up, and have Gemma-3 write Markdown; prompt constraints drop empty items and require business language instead of literal MOVE paraphrases

Outcomes

• ✓ Two programs (13.7 KiB and 13.5 KiB) produced specs of about 6.5 KiB and 6.7 KiB, with retrieved terms matching expectations
• ✓ Established a layered design — static parsing + RAG + tool calls: splitting by subroutine and analysing bottom-up lets a 4B model handle programs longer than its context, with variable descriptions mapped to the right business names

Data flow

Knowledge base (manuals → semantic split → embeddings → ChromaDB) ⇄ MCP ⇄ COBOL source → preprocessing → ANTLR4 → AST → PERFORM call graph → topological sort → Gemma-3 per-subroutine generation → functional spec .md

System architecture

Responsibilities

• Designed and built the main business service (Gin + MongoDB) in router/controller/service/model layers — 17 modules and about 100 endpoints covering models, entities, equipment, data dictionaries, templates and scenario versions
• Designed simulation-engine lifecycle management: dynamic port allocation, scenario export as engine input, process launch and registration via os/exec, goroutines that detect exit and reclaim resources — supporting several simulations in parallel
• Implemented multi-user scenario co-editing with a WebSocket hub and per-task rooms: separate read/write goroutines with heartbeat, a flattened scenario-tree index cache, and incremental command broadcast
• Built the operator service: scenario JSON → external simulation-engine XML conversion (3,000+ lines) and GeoJSON area/intersection/union computation; per-frame situation recording with pause, speed control and timeline replay
• Wrote the engine console in Go: connects to the scheduler over TCP, relays engine commands, and exposes HTTP endpoints to start, pause and resume a simulation
• Built a radar-signal generation service with Tornado + SciPy (chirp pulses, clutter, echoes), generated asynchronously on background threads and deployed with Docker

Data flow

Scenario editor (Vue + Cesium) ⇄ WebSocket co-editing → Gin main service → operator service (JSON → XML) → engine console (TCP) → simulation engine → situation feedback → 3D display / replay

System architecture

Responsibilities

• Built the platform-integration layer over ZeroMQ (REQ + DEALER) and Protobuf, wrapping 38 interfaces — simulation control, unit/task/damage queries, routes, take-off, grouping; a runner drives the observe → decide → command loop at a fixed step and scores the outcome
• Red defence agent: staged decision-making — perception, information fusion, missile/gun/laser fire allocation, strike verification. Blue attack agent: multiple courses of action, drone-squadron launch and saturation strikes
• Deployed and evaluated 16 entered agents (rule-based, ML, deep networks), resolved dependency and runtime problems, dissected representative strategies and gave a tech talk
• Agent state is published over MQTT; built a PyQt5 live monitor with tabs for red/blue units and damage, plus per-round offline dumps for review

Data flow

C++ wargaming platform ⇄ ZeroMQ + Protobuf ⇄ runner (observe → decide → command) → red/blue agents → MQTT → PyQt5 live monitor

System architecture

Responsibilities

• Worked on the traffic pipeline: capture probe → Kafka → protocol-parsing engine → PostgreSQL; used Redis key-expiry events for 10-second flow aggregation and batch inserts, sharply cutting database writes, and Redis TTLs for device-offline detection
• Implemented the allow-list detection engine keyed on 5-tuple, MAC and protocol function code, with a bitmap that separates "behaviour anomalies" from "content anomalies"; deep rule matching on IEC104 ASDU and OPC interface/function codes, plus automatic allow-list learning over a chosen time window
• Built the Django REST endpoints for asset management, allow/deny-list policies, alerts and reports; Celery Beat jobs for report export, policy backup, expired-data cleanup and probe health checks
• Integrated Suricata, parsed unified2 alerts and loaded nearly a thousand (996) industrial-vulnerability signatures for deny-list detection
• Synced data to a Pilosa bitmap index through Kafka Connect (Debezium CDC) to back traffic statistics and multi-dimensional queries
• (Customised edition) Parsed IEC 61850 SCD files to generate substation IED communication baselines and topology automatically

Outcomes

• ✓ Protocol parsing is CPU-bound, byte-level work: the parsing library is written in Cython, compiling hot paths to C extensions — near-C speed with a Python interface for upper layers; only the .so is shipped, which also protects the core code

Data flow

Switch mirror port → probe (Scapy) → Kafka raw_packets → Cython parsing engine → Redis 10 s flow aggregation → PostgreSQL → allow-list detection → alerts → Django + Vue; Suricata runs deny-list detection in parallel; PostgreSQL →(Debezium CDC)→ Pilosa

System architecture

Responsibilities

• Built the Ray-based distributed algorithm scheduling platform (owner): orchestrated 10+ capabilities as actors — adult-content and violent/terror detection, OCR, TV-logo and logo detection, face search, video copy detection, cross-modal search
• Message reliability: Kafka auto-commit off, commit after each processed batch (at-least-once); puller actors start 1:1 with the partitions the client discovers
• GPU slicing: fractional resources with num_gpus=0.2 let one card host 5 model instances in parallel (single node, 4× Tesla T4)
• Designed a three-layer abstraction — message, routing, model: the message layer re-batches inference input as downloads finish with ray.wait; the routing layer intersects the subscribed algo_id set with each model's capabilities; adding a model means adding one actor and touching nothing else
• Wrapped C++ algorithm libraries as Python extension modules with SWIG 4.0 (owner): images are passed as numpy memory pointers after a C_CONTIGUOUS check, removing cross-language copies
• Contributed to the Huawei Ascend Atlas ACL inference SDK (C++11 / CMake): device and stream management, FFmpeg demuxing with DVPP hardware decoding; migrated vector search from AscendFaiss to Milvus; adapted to three CANN driver releases; found and fixed index-deletion failures in multi-chip search caused by an OpenMP thread-count mismatch
• Dual-platform model delivery: PyTorch → ONNX (opset 11) → TensorRT 7 INT8 (for Tesla T4) and Ascend ATC .om models, so one business codebase runs on both NVIDIA and Ascend; 17 model packages maintained
• In-house OCR pipeline for sensitive content: PSENet detection + CRNN (LSTM) recognition + angle classification, with results matched against a ~20,000-word dictionary using a DFA algorithm

Outcomes

• ✓ 10+ algorithm capabilities served from one scheduling platform; the GPU and Ascend SDK archives together hold 17 model packages
• ✓ Unified the vector-search backend on Milvus so the GPU and Ascend lines share one retrieval interface

Data flow

Kafka tasks → pullers ×N (1:1 with partitions) → message layer → routing layer (algo_id ∩ model capability) → model actor pool (×5 per card) → result Kafka; C++ SDKs attached via SWIG; gallery data → Milvus + MySQL

System architecture

Responsibilities

• Tornado service accepting text, image and video: video is sampled every 25th frame, encoded by a cross-modal two-tower model into 512-dim features, with frame features averaged
• gRPC lookups against Milvus return TopK candidates, a multi-condition logic engine (AND / OR / NOT / contains) does set operations, RRF reciprocal-rank fusion re-ranks, tags are attached in batch, and Redis serves as a second-level cache
• Four performance iterations: single gRPC queries → concurrent batch API → logic filtering first → multi-threaded sharded queries (100 per batch) with np.dot batch matrix products; gallery lookups went from N serial gRPC calls to ⌈N/100⌉ concurrent batches
• A sibling offline product handles video object filtering: frame extraction → SDK 512-dim features → gRPC search in Milvus → post-processing → Docker delivery

Outcomes

• ✓ Delivered 3 customer-specific offline editions in total (Docker images)
• ✓ Replaced per-item similarity comparison with a single matrix dot-product, with threshold cut-off for early exit

Data flow

Three-modality input (text / image / video) → frame sampling → cross-modal encoding (512-dim) → gRPC · Milvus → logic engine → RRF re-rank → tagging → Redis cache → JSON

System architecture

Responsibilities

• Data-collection module: integrated crawler services over RESTful APIs, ran collection tasks from the UI, stored results in MongoDB
• Data-processing module: exposed OpenCV image ops and FFmpeg video ops as Tornado APIs, operable from a Vue UI; designed resumable chunked uploads
• Model-defect analysis module: GUI for initial labeling and human review; computed PR metrics across datasets/models and auto-generated PDF reports
• Worked across frontend and backend, including page fixes and UI improvements

System architecture

Responsibilities

• Two-stage recognition: Mask R-CNN (R-101-FPN) finds students, ResNet18 classifies behaviour into 6 classes (confidence below 0.95 becomes "uncertain") — more accurate than one detector doing both, and changing the behaviour set only means retraining the small classifier
• Real-time analysis: OpenCV reads the RTSP stream and drops frames whenever the video timestamp lags wall-clock time; one GPU process per class, with an nvidia-smi free-memory check before launch
• Tracking and identity: wrote my own tracker using bidirectional overlap over a 5-second window; face_recognition (dlib) 128-d features are matched to the class face gallery, and the most frequent ID within a track wins — so identity survives head-down and side-face frames
• Live alerts for slumping / turning away: the student's face crop (base64) goes into the alert; latest results and per-frame history are stored separately in MongoDB
• Tornado backend: accounts with three roles, alerts and reports at four levels (student, class, grade, school), a WeChat mini-program API and an open API for third-party platforms; designed API field protocols against the product manager's prototype
• Analysed behaviour data with scikit-learn and produced weekly, monthly and term reports per student, pushed to parents' WeChat mini program
• Internal labelling and training platform: video trimming → Mask R-CNN pre-labelling (labelme JSON) → human correction → cropping → dataset building → ResNet18 training (resumable, loss/accuracy curves) → validation; the whole-lesson offline analysis was the prototype of the real-time algorithm
• Delivery: compiled the core tracking algorithm to .so with Cython, ran under supervisor and used a licence dongle, deployed on-premise at several customers

Outcomes

• ✓ One codebase serves three customers' custom interfaces, a full third-party open API and the WeChat mini program
• ✓ Turned data production into web tools: model pre-labelling cuts manual work so non-algorithm staff can take part in labelling

Data flow

Camera RTSP → OpenCV read / frame drop → Mask R-CNN detection → ResNet18 behaviour classification → face recognition + tracking / voting → MongoDB → Tornado pages / live alerts / WeChat mini program

System architecture

Responsibilities

• Stats-engine backend (Tornado, ~1,650 lines, 21 routes): lists, details and history trends for classes, teachers and students, teaching-attitude time series, model-class TopN
• Elasticsearch queries: bool + terms + range, nested avg sub-aggregations under terms (per-class teacher happiness probability, word count, speech rate), scan/scroll for large result sets; CV and ASR results merged and sorted in the application layer
• Admin frontend (vue-element-admin 4.4): role-based dynamic route permissions, data dashboard, class/teacher/student lists and details, rankings, trends, filters; ECharts visualisation, Docker and multi-environment config
• Part of the algorithm analysis: combined speaking share, fluency and positive emotion to find ~30-second "highlight" clips, flagging noise, courseware audio interference and parent presence, for the progress report sent to parents

Data flow

Page-capture SDK → OSS → ASR / CV / VAD workers → Elasticsearch → analysis / stats engine → admin portal + MQTT live alerts

System architecture

Responsibilities

• Inference flow: random frame sampling → YuFaceDetectNet detection with 5 landmarks → affine alignment → occlusion / pose-blur / age models filter → ResNet18 features compared by cosine similarity; the youngest face (< 17 years) is taken as the student, excluding parents
• Mixed-framework inference: PyTorch for detection and features, MXNet Gluon for occlusion, pose and age, all on CPU; iterated from v1.0 to v3.4 (dlib 68-point + Laplacian blur check → new models and alignment → three quality models → rewritten full-gallery search and video validation)
• Features live in Redis with global, incremental and per-person search; a full pairwise pass is estimated at about 40 million comparisons per the code comments
• Commercial licence control: call counts, expiry and daily quota are AES-encrypted in Redis, SETNX gives a distributed lock, and the device SN is verified; the core module ships as a Cython-built .so
• Config is read in the order command line > environment variables > JSON/YAML; Docker deployment pulls model weights from OSS at build time

Data flow

Video → random frames → face detection / alignment → occlusion, pose, age filters → feature extraction → cosine matching / Redis feature store → verification / search result

System architecture

Responsibilities

• YOLOv5s as the main detector (7 UI-element classes) with rules to decide the screen type; labelled 10,000+ screenshots and 1,000+ like/comment icons myself, with k-means anchor boxes
• Post detail: PaddleOCR (PP-OCR, DB detection + CRNN recognition) reads post time, ad copy and poster ID by layout position; QR decoding; the bottom-right is cropped, like/comment icons are detected with YOLOv5 and the digits read by a 3-layer CNN
• Tornado + PyTorch + PaddlePaddle service: token and MD5 integrity checks, nanoid request tracing, originals in OSS, every result and latency written to Elasticsearch for debugging production errors
• Upgraded the PP-OCR model release, improving poster-ID and post-time recognition

Outcomes

• ✓ Digit-reading 3-layer CNN: 98.95% train / 98.25% validation / 84.45% test accuracy

Data flow

Screenshot upload → token / MD5 check → YOLOv5 element detection and classification → PaddleOCR + QR + like-count reading → JSON; all results to Elasticsearch