15

I've been hearing this claim being bandied about quite a lot and I'm wondering if there's any substance to it. Apparently the details are in a certain thread on Talkchess, but it's not viewable to non-logged in readers, and it won't let me register. There are no details about this on the Houdini Wikipedia page either.

Can anyone give a summary of the allegations and the evidence for (and against) it?

4 Answers 4

12

Yes.

The search, evaluation, etc., are almost identical to Stockfish, with the notable exception of being rewritten in Dutch. A number of Stockfish developers, including former maintainers, have noted as such. To hide its origins, code was included that multiplied the node count by 8/7. Exploiting modular arithmetic tricks and archived Houdini games, this exact code was shown to be present in the release version of Houdini. The alleged performance improvements over Stockfish go away after removing this trick.

The leaked source code was compiled by Ed Schroder and found to have a 1:1 correspondence with Houdini in a vast number of test positions, while others, like Simon Guenther, have found that the leaked code behaves identically in the start position even up to high depths.

There were changes, but Mark Lefler, developer of Komodo, noted that a number ofaa changes were simply adding features reverse engineered from Komodo. In fact, in one file, evaluatie.cpp, you can spot the naming conventions used by the Hex-Rays decompiler.

Other features were unique, but minor. For example, the licensing system was not from Stockfish. However, the code came from TurboPack and was translated into C++, so it's not clear that that was original work, either.

Adjusting for the time of release, the version of Stockfish on GitHub at the time of Houdini 5's release was more or less equal in strength to Houdini 5. Later, when Houdini 6 was released, this was also true. (Source: nextchessmove's elo data + CCRL data)

In other words, Houdini did not notably build upon Stockfish in any meaningful way. The main reason it looks different is because of the Dutch translation and the fact that Stockfish has evolved significantly since Houdart last touched the Stockfish codebase.

10
  • Is there an explanation for why Houdini looked so different then? For example, I notice a lot of its games which end up drawn which it's playing White, its eval drops to slightly below zero before going back up to 0.00, which Stockfish doesn't do. Houdini was also the strongest engine on CCRL for a few months, I think.
    – Allure
    Apr 12, 2020 at 20:43
  • Stockfish has new official versions out every ~six months. While there's no official cadence, releases come slowly while the Stockfish version on the GitHub repo is constantly updated. At the time Houdini 6 was put on CCRL, Stockfish was at least 35 elo stronger than CCRL's version, based on NCM data. The source code looks nearly identical to SF's, minus the variable name changes. However, since it was based on an intermediate Stockfish version, incorporated Komodo code, & had small tweaks, it played a bit differently from any official SF releases. Apr 12, 2020 at 20:52
  • Hmm I remember a time when Houdini was the strongest engine, better even than Stockfish dev - e.g. right around the time when it won TCEC Season 10. How is that possible if it is almost identical to Stockfish?
    – Allure
    Apr 12, 2020 at 21:50
  • 1
    Houdini at one time, was the world's strongest, it consistently top Stockfish in all ranking lists and TCEC. How would you explain it?
    – SmallChess
    Apr 13, 2020 at 8:13
  • 1
    @SmallChess It did not consistently top Stockfish. Stockfish lost season 10 because it was not configured to punish weaker engines, as Komodo and Houdini were, so it did not promote past Division P. For that season, it earned the name "Drawfish" despite playing the most precise chess. Prior to the rise of Fishtest, Houdini beat Stockfish by building on top of Robbolito. This is well documented. (See Richard Vida's analysis.) After Fishtest, Stockfish's dominance became insurmountable. Rating lists generally don't use the development versions of Stockfish, which could easily be >40 elo stronger Apr 13, 2020 at 23:33
4

The alleged source code might have been published online. I took a quick look into it, it looked like a valid strong chess engine. It was not a clone of Stockfish in the sense that the codebase was quite different, certainly not a 10 minutes copy-and-paste. It would've taken even the best talent months (or years?) of hard work to develop it, so it doesn't look like a prank. It certainly looks like the Houdini source code. I can't find anything in the code that says otherwise.

My personal feeling is the Houdini author must have studied Stockfish's source code very carefully. But ... honestly who wouldn't? Komodo is famous for tracing every single bit in Stockfish and apply/improve the ideas in their own engine. I believe everyone who is serious in chess engine programming would treat the Stockfish source code like a bible.

I feel Houdini was unsurprisingly sourcing information/ideas etc from Stockfish and Robolitto, but it was not a clone more like a genuine efforts to improve old ideas and develop new methods.

The definition for cloning is confusing. Chess engine authors believe a "clone" happens if an idea (or some code) copied from Stockfish. The general public would think a "clone" is a direct copy-and-paste from Stockfish (maybe just change the program name in UCI?).

We need to work out the definition.

PS: the code looked like a stolen product:

enter image description here

1
  • 3
    Apparently Houdini's nodes per second is inflated by 7/8, which can be verified by examining its actual speed. In other words, Houdini never outputs a nodes per second which is 7 modulo 8 (and it indeed never does). Any idea how this can be explained other than as dishonest reporting?
    – Allure
    Apr 14, 2020 at 5:02
4

A user on talkchess.com (by the name of cucumber) claims to have come into possession of Houdini's source code, and has noted a number of similarities between it and Stockfish's source code. In particular, they note that the Makefile contains the phase 'to build Stockfish' and they link a screenshot of a search function which appears to be Stockfish's, but translated to Dutch:

initial post by cucumber

It is unclear if the source code they have is actually Houdini, though a few contributors to the thread allege that the compiled source closely mirrors the behaviour of the actual engine binary.

Additionally, cucumber claims that they obtained the source code illegally. post clarifying how the source was obtained

It seems that all claims here hinge on whether or not cucumber actually has Houdini 6's source code. Since they have stated their intent to not release it publicly, this will be difficult to verify.

5
  • I thought it was published?
    – SmallChess
    Apr 12, 2020 at 14:55
  • "I do not intend to publicly release any of Houdart's original code. I also do not intend to share all of the source with anyone privately. Only what seems necessary." - cucumber.
    – konsolas
    Apr 12, 2020 at 14:58
  • It's possible that it was published at some point in the 38 page (as of now) thread, in which case I must have missed it.
    – konsolas
    Apr 12, 2020 at 14:59
  • The source code has been published on Github, and compiling that code creates an executable that's basically Houdini 6.As to the legality of obtaining that code, it likely depends on your legal jurisdiction. The repository was publicly accessible on the internet.
    – HTTP 410
    Apr 12, 2020 at 17:41
  • 1
    Thanks for answer, but can you link to a screenshot instead of that thread since it's not accessible without logging in?
    – Allure
    Apr 12, 2020 at 20:44
4

I saw the code and there is no doubt that code was stolen form Stockfish Here's a small snippet that I saved. I no longer have the code since I found out it also includes code stolen from komodo ( reversed engineer). There are many many examples just like this,

The one of the right was from Houdini, the one I the left is from the Stockfish repository from a few years ago, not current dev Stockfish - commit begins with in "...cd8dc0a" https://github.com/official-stockfish/Stockfish/pull/706/commits/cd8dc0a1016cefe332f41fd7645b4b8462b1e466

I was able to build an exact clone of Houndin 6.03 from the entire source that is on the right. This is just one example - that are many snippets just like this one. It provided exact node count and pv when one performed "go depth xx" on any position with Houdini 6.03.
enter image description here

1
  • How do you know whatever you built is an exact clone of Houdini 6.03?
    – Allure
    Nov 9, 2020 at 5:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.