Skip to content
J.R.
日本語 EN 中文

02 / 05 · 2020.08 — 2022.08

People's Zhongke Intelligent Technology Co., Ltd.

Owned algorithm scheduling and the cross-language layer of a content-safety platform: Ray to orchestrate 10+ capabilities, SWIG to bring C++ SDKs into Python, a share of the Ascend Atlas inference SDK, and a cross-modal retrieval product shipped offline on my own.

Role
Algorithm Application Engineer
Industry
AI × content safety
Location
Beijing
Team
15 people
Tenure
2 yrs 1 mo

Key results

  • 10+

    Capabilities orchestrated together: adult content, violence/terror, OCR, TV logo, logo, face search, video copy detection, cross-modal search and more

  • 5 per card

    Fractional GPUs (num_gpus=0.2) let one card serve 5 model instances (single node, 4× Tesla T4)

  • 3

    Customer-specific offline editions delivered (Docker images)

Projects here

PROJECTS · 3
Seal-detection output: the stamp on a sample document is boxed with its confidence (0.95)

2020.08 – 2022.08（2 yrs） · Platform / algorithm backend (Python / C++11) · Team 15

AI content-safety algorithm platform

An AI system modelled on leading content-moderation products: it monitors and analyses public media and offers review capabilities to government users and partners. It exposes 10+ algorithm capabilities by subscription across images, video, audio and text. I owned the Ray scheduling platform and the SWIG cross-language layer, and contributed to the Ascend Atlas ACL inference SDK and the dual-platform model toolchain.

  • Built the Ray-based distributed algorithm scheduling platform (owner): orchestrated 10+ capabilities as actors — adult-content and violent/terror detection, OCR, TV-logo and logo detection, face search, video copy detection, cross-modal search
  • Message reliability: Kafka auto-commit off, commit after each processed batch (at-least-once); puller actors start 1:1 with the partitions the client discovers
  • GPU slicing: fractional resources with num_gpus=0.2 let one card host 5 model instances in parallel (single node, 4× Tesla T4)
  • Designed a three-layer abstraction — message, routing, model: the message layer re-batches inference input as downloads finish with ray.wait; the routing layer intersects the subscribed algo_id set with each model's capabilities; adding a model means adding one actor and touching nothing else
  • Wrapped C++ algorithm libraries as Python extension modules with SWIG 4.0 (owner): images are passed as numpy memory pointers after a C_CONTIGUOUS check, removing cross-language copies
  • Contributed to the Huawei Ascend Atlas ACL inference SDK (C++11 / CMake): device and stream management, FFmpeg demuxing with DVPP hardware decoding; migrated vector search from AscendFaiss to Milvus; adapted to three CANN driver releases; found and fixed index-deletion failures in multi-chip search caused by an OpenMP thread-count mismatch
  • Dual-platform model delivery: PyTorch → ONNX (opset 11) → TensorRT 7 INT8 (for Tesla T4) and Ascend ATC .om models, so one business codebase runs on both NVIDIA and Ascend; 17 model packages maintained
  • In-house OCR pipeline for sensitive content: PSENet detection + CRNN (LSTM) recognition + angle classification, with results matched against a ~20,000-word dictionary using a DFA algorithm
  • Python
  • C++11
  • Ray
  • Kafka
  • Protobuf
  • gRPC
  • SWIG
  • CMake
  • PyTorch
  • ONNX
  • TensorRT
  • 昇腾 ACL
  • Milvus
  • Redis
  • MySQL
  • MongoDB
  • Docker
  • FFmpeg

System architecture

L5 Delivery Product delivery Online review SaaS · Kafka return CDF cross-modal retrieval (offline image) Standard open API · signing / callback L4 Scheduling Ray algorithm scheduling platform Kafka consumer actors ×N MsgWorker · message layer InferWorker · routing layer Model actor pool ×8 L3 Bridge SWIG cross-language wrapper .i interface file _wrap.cxx → .so numpy pointer, zero copy pyporn / pyivcpd … L2 Inference C++ inference SDK acl_manager decoder · FFmpeg + DVPP inference / pre- & post-processing Retrieval index L1 Hardware Heterogeneous compute NVIDIA T4 · CUDA · TensorRT 7 INT8 Huawei Ascend Atlas · ACL · DVPP Side Toolchain Data & model toolchain Crawler collection Frame cut / label parsing pth → ONNX Robustness evaluation
Online inference path Kafka task topic Protobuf / JSON message Puller Actor ×N 1:1 partitions · manual commit num_cpus=0.2 MsgWorker ×3 Message layer · parse & assemble ActorPool.map InferWorker Routing layer · algorithm routing algo_id ∩ capability set batch_imgfile_read remote task Batch download/decode · ray.wait re-batches Model layer · one actor per kind / num_gpus=0.2 per card Adult content porn_actor Violence / terror jzk_actor OCR ocr_actor TV-logo detection taibiao_actor Logo detection logo_actor File-type recognition file_actor Face search face_retrv_actor Video copy detection ivcpd_actor Aggregate results Assemble tags per API spec Kafka result topic Async return Gallery-management path · task.py Kafka gallery topic Add / delete / update Face / video gallery actor Quality gate · feature ingest Milvus vector DB :19530 MySQL metadata :33060
Video / image inference pipeline ① Video / image input URL · file · stream ② FFmpeg demux Static build -fPIC ③ DVPP hardware decode VDEC · bypasses CPU decoding ④ DVPP VPC Resize / crop / convert ⑤ ACL model inference Offline .om engine ⑥ Post-processing Boxes / class scores ⑦ Feature / label output feature_index module ⑧ Milvus index Replaces AscendFaiss In-house base modules · src/base acl_manager Device / Context / Stream lifecycle Multi-chip resource allocation BlockingQueue pipeline Decode and inference threads decoupled Multi-stream throughput preprocessor / detector resizer · cropper · operator Shared pre/post-processing base class Driver support: three CANN releases 20.1.0 / 20.2.0 / 21.0.1 · 60+ perf benchmarks optimising FPS / throughput
Build flow porn.h / porn.cpp C++ algorithm & headers pyporn.i SWIG interface · hand-written swig -c++ -python Generate glue code pyporn_wrap.cxx C++ wrapper pyporn.py Python proxy classes _pyporn.so setup.py build_ext Python-side conventions Memory pointer, zero copy img.ctypes.data + C_CONTIGUOUS check STL container template instantiation std::vector → ImageDescriptorVec Pointer / array helpers shared_ptr access · array alloc / free
Data & training Crawler collection XPath + API dual path Data-processing scripts Frame cut · GIF split · VOC parse Training image pytorch 1.8.1-cuda11.1 Model weights .pth Training output ONNX interchange format opset 11 · batch 32 · constant folding NVIDIA route TensorRT 7 serialisation · INT8 quantisation file_int8_bs32_T4_TRT7.en Ascend route ATC conversion · offline model .om engine file CUDA inference · T4 (cc 7.5) GPU SDK ACL inference · Atlas Ascend SDK Robustness evaluation albumentations Flip / blur / noise Compression / colour jitter PR metrics before vs after precision / recall delta

2020.08 – 2022.08 · Sole developer

Cross-modal retrieval offline product (customer-specific)

An offline deliverable for customer-specific needs: the developer-facing, many-parameter model API is wrapped into one simple retrieval endpoint and shipped as a Docker image, hiding model paths, feature dimensions and index partitions.

  • Tornado service accepting text, image and video: video is sampled every 25th frame, encoded by a cross-modal two-tower model into 512-dim features, with frame features averaged
  • gRPC lookups against Milvus return TopK candidates, a multi-condition logic engine (AND / OR / NOT / contains) does set operations, RRF reciprocal-rank fusion re-ranks, tags are attached in batch, and Redis serves as a second-level cache
  • Four performance iterations: single gRPC queries → concurrent batch API → logic filtering first → multi-threaded sharded queries (100 per batch) with np.dot batch matrix products; gallery lookups went from N serial gRPC calls to ⌈N/100⌉ concurrent batches
  • A sibling offline product handles video object filtering: frame extraction → SDK 512-dim features → gRPC search in Milvus → post-processing → Docker delivery
  • Python
  • Tornado
  • gRPC
  • Milvus
  • Redis
  • NumPy
  • OpenCV
  • Docker

System architecture

Cross-modal retrieval flow and performance iterations
Main retrieval flow Three-modality input Text / image / video base64 or plain text Frame sampling / decode cv2: every 25th frame Video branch only Cross-modal two-tower text.en / image.en Unified 512-dim features Video feature synthesis Average of frame features np.average Milvus gRPC search TopK candidates Multi-condition logic engine AND / OR / NOT / contains Result-set operations RRF re-rank Reciprocal rank fusion 1/(rank_l + rank_r) Batch tagging QueryTags batch TopK tag words Redis cache → JSON Video info / tag 2nd-level cache Tornado async response Four performance iterations · history in API/readme v1 · old Single gRPC query Baseline v2 Concurrent gRPC API Batch request merging v3 · new Logic filter first Filter, then tag v4 · thread Multi-threaded sharded query 100/batch + np.dot similarity

2020.08 – 2022.08 · Systems Engineer · Team 1

CV Data-Processing System

Built to visualize and streamline the data-processing work that deep learning needs, letting team members collect and transform data through a GUI. Scattered personal scripts were consolidated into one system, improving reuse and efficiency.

  • Data-collection module: integrated crawler services over RESTful APIs, ran collection tasks from the UI, stored results in MongoDB
  • Data-processing module: exposed OpenCV image ops and FFmpeg video ops as Tornado APIs, operable from a Vue UI; designed resumable chunked uploads
  • Model-defect analysis module: GUI for initial labeling and human review; computed PR metrics across datasets/models and auto-generated PDF reports
  • Worked across frontend and backend, including page fixes and UI improvements
  • Python
  • Tornado
  • Vue
  • Vue-Element-Admin
  • MongoDB
  • OpenCV
  • FFmpeg

System architecture

CV data-processing system modules and data flow
L1 UI Vue admin UI one UI instead of personal scripts L2 API Tornado API layer RESTful API · task dispatch L3 Three modules Data collection RESTful link to crawler API collection tasks from the UI Data processing OpenCV image · FFmpeg video scripts resumable file upload Model-defect analysis initial labels + human review (UI) import datasets & model paths L4 Output MongoDB collected data Processed images / video cropping, transforms, etc. PR metrics + PDF report generated by the backend

System data flow

Abstract sketch · no internal details

  1. Kafka tasks

    From upstream microservices · Protobuf

  2. Ray scheduling platform

    Message · routing · model layers

    ▲ My part

  3. SWIG layer

    C++ SDK → Python extension

    ▲ My part

  4. C++ inference SDK

    CUDA / TensorRT · Ascend ACL

  5. Milvus / MySQL

    Features · gallery

  6. Result Kafka

    Written back per protocol

Team & daily life

Photos are shared with colleagues' consent, or other people's faces are blurred

What I learned here

In the AI × content-safety industry I saw far more applications of deep learning. On the vision side that meant OCR and NLP; my main line of work was the algorithm program, using Kafka to tie platforms together, with exposure to Docker deployment and cloud-server operations.

I came to understand how to use compute efficiently — model distillation and pruning, hardware-level acceleration through CUDA / TensorRT, DVPP hardware decoding on Ascend — and a few engineering trade-offs: a three-layer abstraction so protocol changes, algorithm upgrades and new models do not entangle; manual Kafka commits for at-least-once delivery; and optimisation driven by measurements, not feel.

TECH STACK

  • Python
  • C++11
  • Ray
  • Kafka
  • Protobuf
  • gRPC
  • SWIG
  • Milvus
  • PyTorch
  • ONNX
  • TensorRT
  • 昇腾 ACL
  • Tornado
  • Redis
  • MySQL
  • MongoDB
  • Docker