Proposed a pipeline to address the visual sim-to-real gap by decoupling physics from visuals. NVIDIA Isaac Sim provides physically accurate robot motion, and OSCAR-2B adds photorealistic video generation on top. From a single simulation episode, the pipeline captures the robot camera frame and joint trajectory, projects 3D joint motion into a 2D skeleton control video using the episode camera intrinsics and extrinsics, and generates multiple visually diverse input scenes in parallel.

Each diverse input frame is paired with the same skeleton and passed to OSCAR-2B, producing multiple photorealistic training videos from one valid simulation run without re-simulating the task. Experiments across different backgrounds and pick subjects, with the skeleton held fixed, showed reliable arm motion tracking; drop physics after object release was the main limitation.

References

Physical AI, Sim-to-Real, VLA Training, Isaac Sim, OSCAR

• Built an end-to-end synthetic 2D data generation pipeline for robot training, including a control-based fine-tuning and inference workflow for deterministic SKU image generation with controlled dimensions, orientation, and physical properties.

  

• Explored 3D reconstruction and generation methods including Gaussian splatting (Apple SHARP, Meta Map-Anything, Depth Anything v3, MASt3R), open-source world models (Hunyuan World Mirror, Hunyuan World 1), mesh generation models (Hunyuan 3D, SAM3D, Microsoft TRELLIS 2), and hybrid mesh-Gaussian methods (SuGAR).
• Built an agentic pipeline to produce indoor scenes loadable in NVIDIA Isaac Sim for pick-and-place training data aimed at VLA models.

References

Physical AI, Gaussian Splatting, World Models, VLA Training, Isaac Sim

• I built Dexter, an agentic pipeline that turns text or images into articulated assets with separate moving parts, joint definitions, and URDF/USD exports for NVIDIA Isaac Sim.
• Open-sourced it on GitHub and Hugging Face. It is now used by a few researchers at NVIDIA.

References

Robotics, Physical AI, Agentic Systems, URDF, USD, Isaac Sim

• Built a character-consistent image generation workflow inspired by LoRA Shop. Cross-attention maps for target tokens identify activated hidden features, and LoRA features are injected at those positions in Qwen-Image and Flux-Dev to preserve identity across generations.
• Built an image-editing inference pipeline inspired by RF-Edit. Cross-attention maps and hidden features for target tokens are captured during noising, and denoising edits only those features so the rest of the image stays unchanged.

References

Rectified Flows, LoRA, Image Editing, Diffusion Models

• Fine-tuned DeepSeek-R1-Distill-Qwen-1.5B with QAT and published the quantized GGUF checkpoint on Hugging Face.

For other benchmark results, or to download and use the model, see the Hugging Face model card.

References

Model Research, QAT, LLM Inference

I contributed to a white paper that benchmarks LLaMA-3.1-8B inference under a controlled decode workload: single-token inputs generating 512 output tokens, with sweeps over batch size and GPU clock frequency. The experiments report tokens per joule, tokens per second, end-to-end latency, and time-between-tokens, and compare three serving goals. Energy-efficient serving peaks at large batches with moderate clocks (batch 32 around 1000–1400 MHz). Low-latency streaming favors small batches at higher frequencies (batch 1–2 at ≥700 MHz, with diminishing returns past ~1200 MHz). Throughput-driven serving favors large batches at high clocks (batch 32 around 1400–1600 MHz). The paper also introduces DheyoGPUpd, a telemetry and power-control agent for heterogeneous NVIDIA and AMD GPU fleets.

Download white paper

LLM Inference, Energy Efficiency, Performance Analysis

I wrote a white paper on constraint-aware prompt enhancement for text-to-image models. Prompt expansion often improves image quality but drops explicit constraints such as negation, counts, and attribute requirements. The pipeline starts with supervised fine-tuning, then applies GRPO with structured visual feedback from generated images so expanded prompts stay aligned with user intent.

Download white paper

Diffusion Models, Reinforcement Learning, Prompt Enhancement

• Analyzed dense layer and self-attention matrix multiplications in transformer blocks on NVIDIA H100, deriving FLOPs, IO, and arithmetic intensity for initialization and autoregressive decoding phases.
• Benchmarked latency and throughput across batch size, sequence length, and hidden dimension; mapped results with roofline analysis and published open H100 benchmark data.

References

Transformers, H100, Roofline Model, LLM Inference, Performance Analysis

Under the vision of Raja Koduri

• Architected a PyTorch compiler that leverages torch.compile and torch-mlir to enable efficient deployment of LLM and diffusion models across heterogeneous hardware.
• Engineered an asynchronous queuing system for torch dispatch that enables efficient execution across heterogeneous hardware.
• Developed an optimized garbage collection system for on-device tensor deallocation in torch's eager execution mode during runtime.

PyTorch, Compilers, torch-mlir

• Designed algorithms for efficient matrix multiplication tiling for hardware acceleration.
• Researched CUDA and CuBLAS tiling strategies to enhance tensor operation performance.
• Analyzed batching effects in MLP and Attention layers of transformer architectures to optimize inference throughput.
• Evaluated Deepseek_r1_Distil_Qwen_1.5_b and CogVideoX models, creating optimized inference pipelines for deployment.
• Implemented Llama 3.2 1B from scratch with optimizations for deployment across heterogeneous hardware.

PyTorch, LLMs, Performance Optimization, Model Deployment

Under the mentorship of Satya Uppalapati, with guidance from Wajahat Qadeer, Rajashekar Reddy, and Siva Kumar Vemuri. Thesis submitted to BITS Pilani, India.

LLMs, Academic Research, Master's Thesis

• Examined systematic outliers across various Language Model architectures (LLMs) to evaluate their suitability for deployment on edge devices, including llama7b (both base and chat versions), qwen7b, and tiny llama.¹
• Investigated novel quantization methods for optimizing LLMs including, AWQ, GPTQ, GGUF/GGML.^2,5,6
• Developed a framework for LLM Smoothing, refining model weights using a modified version of OmniQuant that incorporates smoothing into the input of the down projection layer of the attention block and transitions from dynamic to static quantization.^3,4
• Implemented Flash attention type tiling and an online normalization calculator for SoftMax to achieve memory-efficient precise attention mechanism on ARA NNPs.^7,8
• Conducted analysis and pruning of LLM layers using Singular Value Decomposition and Block Importance methods to eliminate less important or redundant layers, thereby enhancing model memory bandwidth. Improved throughput from 2 Tokens/sec to 9 Tokens/sec with minimal decrease in accuracy on lm-eval.^9,10
• Employed LoRA, QLoRA, and LoRA+ techniques to restore pruned models to achieve SOTA accuracy on lm-eval.^11,12,13
• Established a Knowledge Distillation framework using PyTorch's FSDP method for distributed computing across multiple nodes and GPUs, utilizing NVIDIA's A10s, H100s, and A100 GPUs. This framework facilitates Knowledge distillation from a teacher model (e.g., Qwen7B or LLaMA7B) to a student model equipped with Quant and DeQuant (QDQ) stubs, supporting both static and dynamic quantization for QAT of the LLM model.^14,15

References

LLM, QAT, Knowledge Distillation, Finetuning, Pruning, FSDP, Outliers

I systematically explored the impact of different rounding techniques in post-training quantization, focusing on the ResNet50 model. The evaluated rounding methods include Rounding Away from Infinity (RAI), Round to Nearest Even (RNE), and Ada-round Simulator.

Results

References

Quantization, Rounding, Classification, Mathematics

This work focused on optimizing OpenAI's CLIP model by enhancing its Transformer blocks, specifically focusing on the Key-Query-Value (KQV) projection layers through quantization observer analysis. The goal is to improve accuracy with minimal performance impact. Additionally, I investigated the impact of quantization errors in mean, variance, and inverse square root in Layer normalization within the Transformer block, proposing corrective measures for performance enhancement. Collaborating with Durga, we also analyzed systematic outliers in hidden layer features, developing a new quantization computation to mitigate errors arising from these outliers, as illustrated above.

References

Quantization, Outliers, Optimization, VLMs

This research investigated the influence of different observers on the convergence behavior of quantization aware training (QAT) when coupled with various rounding techniques. The observers considered are min_max_observer and moving_average_min_max_observer, and the rounding techniques analyzed include rne_c/c++, rne_python, and rai. The evaluation is conducted across multiple epochs to understand the dynamics of convergence.

Results

References

Quantization, QAT, Model Training, Rounding

I developed an innovative function approximation for efficient ASIC processors, specifically targeting the inverse square root function. Our novel algorithm revolutionizes its application in normalization layers within neural networks, supporting both powers-of-two quantization and scale with zero-point quantization. Compared to existing techniques, our approximation is approximately 2x faster and exhibits a 30% improvement in accuracy. Evaluation metrics, including Mean Squared Error (MSE), Signal-to-Noise Ratio (SNR), and Mean Absolute Error (MAE), showcase significant advancements in both speed and precision.

Results

The diagram above shows the old methods in red and the new method in blue.

The new algorithm's impact on transformer blocks, widely employed in diffusion models, LLMs, and image generation models like Stable Diffusion, is particularly noteworthy. The enhanced precision and computational efficiency contribute significantly to improving inference speeds in Language Models and facilitating the generation of high-quality images in models like Stable Diffusion.

Mathematics, Applied Numerical Methods, Quantization, Algorithms

This work focused on optimizing YOLOv5 accuracy through activation distribution analysis. The quantized model, using Post-Training Quantization (PTQ) without Quantization-Aware Training (QAT), achieved improved precision by selectively offsetting activation functions to claim more bits for higher precision. Subsequent adjustments in the mathematical operations within the network compensated for the changes introduced in the activation layers.

Results

Deep Learning Operators, Object Detection, Mathematics, Quantization

This research developed efficient int8 kernels and mathematical optimizations for the ROIAlign function. Custom ROIAlign kernels ensure accurate region-based feature extraction in computer vision. I addressed the challenges in bilinear interpolation due to quantization errors, emphasizing the importance of preserving precision. Small errors in quantization could result in significant shifts in bounding boxes (as shown in the figure above, where the red dot represented the actual FP32 point intended for the pixel within the red box, but quantization errors would shift the point to a new position as represented by the blue dot, resulting in the selection of the pixel represented by the blue rectangle). This shift caused high errors in the bounding boxes, ultimately resulting in incorrect object detections in YOLO models. I successfully navigated this challenge, achieving a quantization strategy that struck a delicate balance between computational efficiency and precision.

References

Quantization, Deep Learning Operators, Object Detection, Segmentation, Mathematics

This research introduced a novel kernel designed for accurate powers-of-two approximation exponentiation. Leveraging polynomial fitting and the Newton-Raphson method, our approach (along with Aditya) optimized the computation of exponentiation, offering a balance between precision and efficiency. The kernel's versatility extends beyond traditional exponentiation applications to include the SoftMax function.

e^x = 2^(x/ln(2))

• Precalculate 1/ln(2)
• Multiply this constant by your argument (1 multiplication)
• Use binary shifts to raise 2 to the integer portion of the power (assumes exp+mantissa format)
• Approximate and quantize the fractional part.
• Adjust based on the fractional power-of-2 remainder (likely a second multiplication)

References

Applied Numerical Methods, Deep Learning Operators, Mathematics, Kernels

Paper Link · GitHub

Under the guidance of Prof. N. Ruban at VIT Vellore. Published in IEEE Xplore at CICT Conference.

CNNs, Academic Research, Bachelor's Thesis