Publication
AURORA: Automated Unleash of 3D Room Outlines for VR Applications
ACM SIGGRAPH VRCAI 2024 (with patents) · Project Page
Real-time 3D reconstruction and spatial computing for interactive XR: plane segmentation, point-cloud, scene graphs, and furniture detection / segmentation / tracking that rebuild indoor environments from visual inputs. See Part 5.
Part 1. Agentic AI System (Industry)
Multi-Agent Game Generation Platform @ TikTok US
- Spec / Design / Code / Asset / Review agents that turn user intent into interactive WebGL games, powering a “playable TikTok” experience across 700M+ posts and 55M+ creators.
- Code Agent Runtime: LLM inference harnessed with deterministic and sandboxed execution. LangGraph states are checkpointed at super-step boundaries for retries and partial regeneration. Task completion 63.2% → 91.5%, latency -33% (4m35s → 3m05s), tokens ~-50%.
- Agentic Self-Improvement Loop: VLM scoring, tracing, failure analysis, validation and regression testing. Benchmark score 48.2 → 89.5, iteration cycle from 1 week to 8 hours.
- Infrastructure: queue scheduling, rate limiting, model routing, MQ/FaaS async workers and Redis caching under QPS constraints. Go/RPC services serving 8M+ participants, concurrency throughput +200%.
Multimodal AIGC Evaluation & Data Curation for Game Assets @ ByteDance
- Benchmarks, structured query matrices, failure taxonomies, condition labels and positive/negative examples for textures, sprites, UI and animation.
- Mapped Seedream-4.5 capability boundaries (viewpoint/orientation, seamless texture, style consistency, asset usability), formed post-training hypotheses, and turned findings into skill-level conditioning and generation constraints. Asset-quality score 58.4 → 78.2.
Multimodal AIGC Marketing-Asset Pipeline @ Amazon
- Diffusion generation, SAM segmentation, style/color transfer, semantic embeddings and layout composition for banners, icons and product creatives.
- Quality metrics plus filtering / reranking / regeneration, text tokenization and line wrapping, and block-based layout optimization. Production pass rate 40% → 95%.
- Productionized on AWS (S3, EC2, CodePipeline, SNS) for 200+ campaigns and 1M+ assets. Campaign launch time from 2 weeks to 3 days.
Part 2. Multi-modal LLM, AIGC, Human AI Interaction
Azure-based Intelligent Album Management System
AI Programmer for Rocket.Chat @ GSoC 2024
- Open-source LLM coding assistant (Mistral, Llama) serving 50K+ global users.
- LoRA-based SFT of Mistral on code instruction datasets (synthesis, formatting, bug fixing), unit-test pass rate +10%.
- Multi-turn session management, dynamic model routing by context limit / latency / cost, and GitHub actions via REST APIs and OAuth2.
Generative AI / AIGC (text to image, text/image to 3d model)
Gesture Recognition (Feature Engineering, Data Visualization)
Part 3. Computer Vision
2D Object Detection, Segment, Tracking (Ground-DINO, Yolo V8, SAM2)
3D-Acknowledged 2D Tracking Enhancement (bbox centroid learning)
Part 4. Computer Graphics and Animation
Cloth Physics Simulation (Linearly Implicit Euler, Penalty Collision Forces and Sparse Matrices)
Dual Quaternion Skinning and Skeleton based 3D Character Animation
Project Repository
GPU rendering with skeleton blending and dual-quaternion skinning, improving runtime FPS by 20%.
Linear Skinned Model, Blendshapes and Facial Action Coding System (FACS)
Billiard Simulation Game (OpenGL) with AI Reinforcement Learning (Genetic Algorithm)
Interactive 3D illumination rendering web system (WebGL, JavaScript)
Inverse Kinematics Renderer with Convex optimizer (Gradient Descent, Newton’s Method, BFGS)
Helicopter Game with Keyframed Animation and Interpolation
Part 5. 3D Reconstruction & Processing
Real-time Neural 3D Rendering & View Synthesis (Gaussian Splatting, WebGL/GLSL, CUDA)
Real-time neural rendering and view-synthesis systems for interactive XR, built at the Texas A&M Aggie Graphics Group.
From 2d Video to 3d Point Cloud/Nerf/Gaussian Splatting (SLAM, MIPSFusion)
Indoor Layout Generation, Plane Segmentation (Heuristic Learning)
Project Repository · Part of AURORA (ACM SIGGRAPH VRCAI 2024)
Part 6. Machine (Deep) Learning, Data Analytics
Ensembled Machine Learning System for Financial Risk Analysis and Prediction
Basic logic diagram of LSTM
Prediction on the stock trend
Model Interpretation Analysis
Automatic Big Data Analytics Industrial System
