My Takeaways from Data Hack Summit 2026
I attended Data Hack Summit conference last week (5-7 Aug 2026). It was a wonderful session with lots to learn. Here are the main takeaways I had from that session. Sharing in case you might find it useful.
- Industry has turned decisively. AI is no longer about "will it work". It is now "where will it work?" Identify the use case.
- A few months ago, I joked on LinkedIn that MCPs and Agents are a Software Engineer's way of controlling the narrative in an AI powered world. Received some flak for this post. But today after DHS, I am more convinced than ever. The field of applicative AI is now an engineer's playground. Any dedicated/niche knowledge that ML practitioners, Data Scientists have gained in the last decade are now lying discarded. A Data Scientist is now back to his research work. AI Engineer has emerged and is going to rule the roost. Interesting thing is, AI Engineers have not hit the trough of disillusionment yet. I am wondering if the trough is going to be fairly small.
- A lot of new engineering is coming up. Prompt engineering is done and dusted. The new terms are Loop engineering, Harness engineering, context engineering, Graph Engineering and a few more. Unlike Prompt engineering, some of these are actually engineering. My personal favourite is harness engineering. My reading is - this will grow into a new domain all on its own.
- I have learnt about some new job roles getting created. Forward Deployment Engineer (almost everyone knows this), Vibe Coding Debugging Specialist (this one stumped me)
- The keynote speech by former ISRO chairman Kiran Kumar is enlightening and very inspiring. He is a man who pushed the envelope for India and made a difference. His message is simple:
- Failure is assured when you are pushing the envelope. Be always prepared for it. Do everything you can to avoid it.
- I was fortunate to ask him a few QnA. I asked him how he managed to enthuse his team to work on meager shoestring budgets compared to NASA but still push the envelope. His answer is surprisingly simple. "Make the team believe in your vision. Ideas will follow".
- The keynote speech from Srikanth Velamakanni (CEO Fractal Analytics) is a special highlight. Other speakers have elevated the conference to technical edge. He hit it out of the park (as is his habit). The way he captured the trends and presented them in a new light is mind blowing. For the first time in my life, I have experienced the difference in the points of view of an entrepreneur and an expert. Sincere thanks to Srikanth Velamakanni for his insights.
- No one should ever worry about software engineers losing their jobs to AI.
- The "tribal knowledge" of the organisation should be captured and made available to the AI for unleashing AI's true value.
- Open weight LLMs are becoming a force to reckon. They should be seriously considered by all organisations. They might just become the future.
- The three laws of robotics are no longer relevant. We need a new set of guidelines for our agents. Constitutional AI document from Anthropic is a good read in this direction. More work is needed here.
- Alignment problem is the biggest challenge to be solved. (What AI generated vs what was expected)
- SLMs are silently carving out their place for niche, simple AI use cases.
- Ontologies and Knowledge Graphs are the biggest winners this year. A lot of talk is focused on them.
- Speakers were very clear. Text-to-SQL implementations without ontology support is not worth the investment. "complete loss" was mentioned by one.
- Ontologies should not be driven by committees. Enterprise dictionary is an Ontology designed by a committee.
- Automatically generate default ontologies using a powerful LLM. Have it reviewed by the core focus group.
- Do not aim for an org wide implementation. Start little and revise fast.
- Start with a single department/domain. Get the ontology+KG for it.
- Expand to a second department/domain and merge the ontologies.
- Merging Knowledge Graphs is still a technical and research challenge. (Whatever this means).
- KG and Ontology together offer a near 100% explainability for agent's decisions.
- AI Agents have proven themselves to be a viable technology.
- They are struggling with the below bottlenecks.
- A proper use case for individual businesses/organisations.
- Evaluations.
- A manager's challenge is to identify a use case worthy of AI. An architect's challenge is to build and AI system worthy of the use case.
- Widespread adoption demands a machine first access architecture with accelerated transition to API driven architecture and modular micro services.
- This is essential to provide agents with reliable, secure and standardized interfaces to execute autonomous actions.
- Evaluations only check if the result is acceptable (Yes/No). They do not check if the result is the best it can be i.e., optimization.
- Implicit feedback of users should be considered (in addition to thumbs-up/down etc) to evaluate an agent's success/failure
- Signals like how many times was the question modified before the user got his expected result
- Enterprise Agentic AI playbook
- Look beyond the hype.
- After identifying a proper use case, the focus is on implementation, guardrails and Action oriented intelligence.
- As I have suspected many times and argued in private discussions, RAGAS and DeepEval are inherently limited in their applicability. The backbox nature of LLMs is their weakness.
- Relying on LLM-as-a-judge on your own curated golden dataset and harness is unavoidable for any production use case. This is the failure point in the productionisation.
- LLM-as-a-judge cannot be a blind reliance on the judge LLM. Nor can the confidence scores generated by the judge LLM be relied upon.
- Figure out a way to idempotently quantify the hallucination and groundedness of the answers. This is the only thing that can be relied upon.
- Challenge is no longer technology or framework. All frameworks have matured to comparable feature sets. Challenge is in identifying a use case that can leverage the framework
- Governance of AI and Agents should be prioritised before scaling is attempted.
- Be very clear of Human-in-the-Loop vs Human-on-the-loop
- Secure the agents very religiously
- Agents cannot differentiate between prompts, input text and 3rd party ingested data. This is the chink in their armour.
- Sandboxing will only limit the blast radius. Does not avoid it.
- Consider using jq with JSON instead of Python (in a specific talk by one speaker).
- Loop Engineering by Sudalai Raj kumar (SRK as he is popularly known)
- Not everything is a loop. Evaluate what can be a loop and what cannot be.
- Looping for incremental improvements is not a given. Cannot rely on LLMs to decide the incremental steps. This is firmly in the realm of human creativity
- AI SDLC with multi-agents by Rama Ayalasomayajula
- Her talk very closely resembles some of the work I am doing at Autodesk
- Had lot of notes to compare.
- Bringing in the enterprise specific knowledge is the challenge.
- My takeaway: Unless enterprise’s tacit and tribal knowledge is fed to LLMs, the AI generated SDLC artefacts are going to be of limited value.
- Agent Harnesses
- Claude Code is an example of harnesses. iCode and dCode are other examples.
- Recommendation: Overfit the LLM to work with a harness.
- This makes for a much better combo of model+harness and gives better results.
- Model is not the moat.
- With a different harness, same model can result in 6x performance
- Recommended reading: Anatomy of a Harness, a paper on Claude code.
- Explore Agent Development Kit 2.6.2 (???)
- Context rot: If context grows beyond 60K tokens, then model performance degrades.
- Harness Tax: The tokens used by harness every time an instruction is given
- Dynamic tool discovery is needed to address the LLM's limited search space for tool calling
Open LLM Models
- These are beginning to make noise.
- At least a few businesses are taking these seriously.
- This means in the worst case scenario, they will become niche knowledge in near future. Best case, they will become the mainstay in day-to-day agentic work.
Misc
- There are other sessions like the one from Alessandro on how he used his own Telegram chats to train an LLM etc. These are very interesting but I am not including them here.
- My personal favourite moments from the conference
- OpenCV5 python bindings are released.
- Met Autodesk colleagues from Pune who travelled to B'lore for the conference.
- First time, first-hand experience of how title oriented Indian corporate is. During QnA, I introduced myself this year with my job title. The difference is palpable.
- A few attendees were sitting in the last row and busily debugging production issues while the seminar talks were going on. Must be quite a nasty workplace if it forced people to work when they are attending trainings.
- During networking break, a co-attendee complemented me saying I look only 35y old. Looks like my once-a-year visit to gym is sufficient to de-age me.
