# How to evaluate an AI tool before trusting it (what the REA case teaches) Author: Carlos Imlau, AI Product Builder · Certified trainer (CCP) · AI applied to work. Creator of Tapinfy, FalaTuga and Sourmize. Published on 2026-10-17 · Updated on 2026-10-17 URL: https://www.carlosimlau.com/blog/avaliar-ferramenta-ia-hype-rea ## Summary To evaluate an AI tool, read the official source, confirm the repository, look for independent tests and try it on a throwaway project. GitHub stars measure attention, not quality. ## Key points - REA went from 3 points on Hacker News to 250,000 stars between July and October - Famous tools attract fake copies with malware - Test without real data or access to your accounts - In a team, decide who can install what **How do you know whether an AI tool can be trusted?** Not by its fame. Read the documentation and warnings at the official source, confirm you are on the right repository, look for independent tests, try it on a throwaway project with no real data, check that the use is allowed and measure the result on a case of your own. GitHub stars measure attention, not quality. I build software with AI every day and test new tools often. The REA case shows why this care matters. ## What happened with REA? REA (Reverse Engineer Anything) is a reverse engineering tool built on AI agents. It went from 3 points on Hacker News in July to 250,000 stars on GitHub in October. Online, people said it "clones any app". I went through the documents, the code and the independent tests. What I found is in the video: what the tool really does, a sentence that was removed from the README, a fake copy with a trojan going around, and whether that use is even allowed. > Video from the channel (in Portuguese): [De 3 pontos a 250 mil estrelas: o hype do REA explicado](https://www.youtube.com/watch?v=ysA5csfpBxQ) ## How to evaluate an AI tool in 6 steps 1. **Read the README and warnings at the official source.** What a tool promises in its documentation is usually far more modest than on social media. Read the legal and security notices too. 2. **Confirm you are on the right repository.** Famous tools attract fake copies, and some carry malware. Reach the project through the author's official site or profile, not a random link. 3. **Look for independent tests, not demos.** A demo shows the best case. An independent test shows the normal case. 4. **Test on a throwaway project.** No real data, no access to your accounts, ideally in a separate folder or machine. 5. **Check that the use is allowed.** Read the tool's licence and the terms of the service you plan to use it with. 6. **Measure on your own case.** The question is not "is it impressive?" but "does it solve my problem better than what I already use?". ## Why do stars and hype mislead? Because they measure how many people saw it, not how many used it successfully. A tool can gain thousands of stars in a weekend thanks to a bold sentence, a viral video or a promise its own documentation does not back up. ## What does this change for a company? In a team the risk multiplies: one person installing a fake copy or pasting customer data into a dubious tool is enough. A "who can install what" rule should exist before the first tool. It is one of the topics in my [AI training for teams](https://www.carlosimlau.com/formacao). ## Frequently asked questions ### Is an open source tool always safe? No. Open code lets someone check it, but it does not guarantee that anyone did. Fake copies can be open source too. ### How do I spot a fake repository? Be wary of repositories with look-alike names, few contributors, executable files to download and links that do not come from the author's official profile. ### Should I avoid new tools? No. Test them carefully: always on a separate project, before taking any of them into real work. ### Where can I see more analyses like this? On my [YouTube channel](https://www.youtube.com/@carlosimlau) (in Portuguese) and on the [blog](https://www.carlosimlau.com/blog).