Project summary

I'm working on INLAY, a ultra fast knowledge editing method. I'm investigating when should a learned router help a model on using a new update, reason from retrieved evidence, or just keep it's original answer?

A complicated router looks useful without actually improving over a simple fixed policy. I want to measure when a complexity earns it's place and when the evaluation setup makes it difficult to tell.

I have completed ~5k evaluations cases across 2 Qwen model sizes, 3 datasets and 3 evidence formats. The experiments compare situations when the relevant update is made with situations where it has been removed from the retrieved evidence.

The current research state shows room for routing improvements when the update is present but more room under the removal condition. These are results against benchmark reference answers and they don't prove that a learned router can capture those gains.

What are this project's goals? How will you achieve them?

The next stage tests whether learned routing delivers measurable improvements over fixed policies.

I plan to do this in these steps:

1.) Collect training and calibration outcomes under the same retrieval and answering protocol.

2.) Train and calibrate the routing models using designated data splits.

3.) Compare their decisions against simple baselines, reporting performance across multiple datasets and conditions.

How will this funding be used?

The request is for upto $1k GPU compute costs. (If someone can contribute through modal gpu credits, then that also works for me).

Who is on your team? What's your track record on similar projects?

Working on this solo. I have implemented the evaluation pipeline, completed the initial experiments and prepared analysis and artifacts.

Project code: https://github.com/Aditya-PS-05/INLAY

What are the most likely causes and outcomes if this project fails?

The learned router may not outperform a fixed policy. It might happen that improvements depend strongly on the datasets, scoring methods or disappear after closer review.

Those would still be a useful findings. They will clarify where routing complexity is justified and where benchmark results overstate it's value.

The independent semantic review remains outstanding.

How much money have you raised in the last 12 months, and from where?

I have spent over $3k on gpu compute for this research. They were aws credits, I got from YC.