Engram is used in production at House of Travel. An evaluation on your own codebase should establish what evidence it returns and where it helps your team.
A mixed Git and TFVC software estate, with more than 150 repositories reported in Engram’s existing production reference.
This is a deployment reference, not a controlled performance study. The repository count is an existing published deployment statement, not a new inventory. Engram’s support for 14 languages describes the product’s extraction coverage; it is not a measured language census of this customer.
Our public library-catalog example records the source revision, Engram index timestamp, exact MCP calls and returned evidence. It demonstrates change context, retained history and onboarding on a small corpus.
The captured review matched one submitted file and reported nine affected symbols, while flagging a truncated response and incomplete test reachability. The practical result is a set of source locations to inspect, with the gaps visible.
Those counts describe that snapshot. They establish neither production-scale completeness nor a speed, savings or accuracy multiplier.
Inspect the three workflows →The paid pilot is US$3k for 60 days, credited on conversion to an annual license. Agree the repository set, installation work, support and success criteria during scoping.
Our optional Engram pilot workbook is a custom template for comparing a few real tasks with your normal tools and with Engram. Record the time taken, check the answers and note what was missed. It is not an industry standard, certification or validated benchmark; adapt it to the questions your team wants to answer.
Open the workbook in Excel or import it into Google Sheets. It includes a short setup guide, editable fields for each task, and automatic time, precision and recall comparisons. Keep your completed copy for your team’s review.
Choose representative change, history and onboarding tasks. Record the source commit, Engram release, indexed revision, enabled features and model settings. Have maintainers prepare an expected evidence set before participants start.
Use the team’s normal tools for the baseline and Engram for the assisted workflow. Keep access and task scope the same. Counterbalance task order across participants to reduce learning effects.
Record elapsed task time, correct evidence found, important omissions, incorrect links and unanswered questions. Have a maintainer review the answers. Record indexing cost and model usage separately from query time.
| Task | Evidence to retain | Useful comparison |
|---|---|---|
| Review a change | Matched files, callers, contracts, source links, truncation and test-reachability flags. | Time to a reviewed impact summary; correct and missed dependencies against the expected set. |
| Explain a decision | Commit, ticket, review and rationale citations; retained-history boundary and unanswered questions. | Time to a supported explanation; unsupported assertions and missing evidence. |
| Orient an engineer | Module map, entry point, representative flow and the source locations visited. | Time to explain the selected flow accurately; maintainer corrections required. |
Report the number and spread of runs, not just the best example. With an agreed expected set, calculate precision as correct returned items divided by all reviewed returned items; calculate recall as expected items found divided by all expected items. Keep unknown coverage and incomplete traversals visible.
Publish results only with permission from the owner of the source and measurements. Include the baseline, corpus, task definitions, configuration and limitations so another team can assess the comparison.
Bring a representative repository and the engineering decisions you want to support. We’ll agree an evaluation scope together.