> The recommendations suck, the lists suck — it’s like, 100 lists telling me to read The Handmaid’s Tale and Harry Potter.
I had the same experience with GR and also Amazon.com which constantly peddles the vampire romance books when I am looking for recommendations for horror/fantasy. Both Amazon and GR strategy make sense because best-selling books sell the best, so they should recommend them to increase profits. However, it does suck being a reader looking for new book suggestions.
I've spent a good deal of time making my own book recommendation algorithm which has been working well for me for the last two years. [1] Through it I've discovered old authors I didn't know (Ted Chiang, Clive Barker) and new authors which I wouldn't have noticed (Scott Hawkins, China Mieville). Of course, it always helps to get recommendations from friends with similar tastes, too. :)
As far as I can tell, all recommendations lists fail compared to freely-available experts.
Like, if you like literary fiction, go through the Pulitzer prize for fiction, and just read. (I'm not even half way through that list, but everything I've read on it has been really, really good.) - there's all sorts of awards for smaller niches... the nebula, the Hugo, etc...
(Actually, that's a question. What is the award for the romance genre?)
I... personally don't understand why people even try to automate making better recommendation engines when highly skilled and respected experts are already doing it for just about any niche, and releasing the results for free.
My preference is to rely on the judgement of reviewers whose taste kind of matches mine.
I'm into SF and Fantasy. In the past, I would read the reviews featured in Locus magazine by the various reviewers. Nowadays, I occasionally read Locus but also reviews from other places like tor<dot>com, the Magazine of Fantasy and Science Fiction and Interzone magazine.
> My preference is to rely on the judgement of reviewers whose taste kind of matches mine.
I wish there were a website where I could put in all of my favorite video games, books, movies, anime, etc. And it would recommend me things based on what people with similar tastes liked. Then, I could try out the recommendation and then either like or dislike it.
I would slowly acquire a recommendation network of people like me, effectively crowd sourcing the content-finding to a bunch of clones!
That's FilmAffinity, for movies. It's pretty popular in the Spanish speaking world.
You vote movies, it calculates what it calls your "soulmates" (people who voted similarly) and then you get recommendations Of movies based on what those people loved, which you can filter by genre, era, etc. You can recalculate your soulmates anytime, and there are some options to tweak the algorithm.
You can also organise movies into lists (kinda like music playlists) and those are public, so if my preferences match user X and they happen to have a list titled "movies I enjoyed this past year" I can just check that.
That's essentially what https://www.senscritique.com/ does in the french speaking world. You rate movies, books, comics, series, music, then it will suggest "éclaireurs" (scouts/recon) to follow, finally you will be advised to see/read/listen what they love and you do not know.
This used to be what Last.fm did before they turned into a MySpace-like mess. Reviews are great but in a world of access to a monumental volume of content, a way to discover obscure things is still difficult to find, and apparently, to monetize. When Netflix started streaming, they tried the long tail approach, but soon changed to only licensing a few things and then serving them up in endless configurations, which must be far more business-friendly for them.
https://www.anime-planet.com/ does recommendations like that on a per-show basis (if you liked [this] then you may like [this], [this], [this]...), the relationship recommended, voted, and commented on by other viewers.
I think Steam is working on something similar, with recommendation based on what you're playing (https://store.steampowered.com/recommender). Well it recommends only games on the store, so that's a lot of games off the table right from the start.
RateYourMusic can do exactly this; find some users with similar tastes (you can see who rates an album as what score), and add them as friends (privately, i.e they won't be notified when you add them). Then you can make lists of how your friends collectively rated music. You can also find site-wide charts for specific genres.
makes sense. I've been reading Gaitskill's "little hammer" - it's a collection of essays, but... like half of them are book reviews, so now I'm pausing to read some mailer. For that matter, it is pretty common for me to read books because they are mentioned in the front matter of other books.
Because we don't always like awarded books. I read "The Three Body Problem" a couple of years ago which had won a Nebula award and found it to be mediocre at best and much overhyped. Last year I read the "Neuromancer" which had won both a Nebula and a Hugo and was for me one of the worst SF books I've ever read. And then there are books which are brilliant, but have never won an award, like "The Martian".
A book like the "Neuromancer" really needs to be understood in it's time. If you read it now, it will look very clichéd. But that's because it introduced and popularized those very clichés!
A lot of really significant cultural works have become so absorbed in our common culture, and sometimes improved upon, that the original starts to look cliché and even naive. Is there a name for this effect?
I read Neuromancer in the early 90’s and thought it was very cliched at the time. Granted that’s ~10 years after publication, but it barrows heavily from earlier works.
I suspect people like it for the same reason they liked their first Anime, it’s an unusual style that seems very original unless you have been reading other stuff written in the same vein by say Philip K. Dick.
The focus on cyborgs a year after The Six Million Dollar Man TV show kind if shows how much a product of the times it was. But, that’s the surface.
The way it portrayed both hacking and brain machine interfaces was wildly off base and basically copied from other science fiction. Virtual reality for example goes back to 1933. Main character being a druggy is fairly common in that time period, again not a big deal. As is copping tone from other works etc.
All the big stuff is forgivable, but he also copies little things like replacing liver and kidneys to better filter the blood and thus prevent someone from getting high / poisoned etc. Sounds good, but blood takes around a minute to circulate and most of it does not hit either on the way. It might reduce how long someone stays high or improve their chances when poisoned, but it’s really not enough to prevent it.
Granted I prefer hard sci-fi, but the novel’s focus is really on style over science fiction. It’s IMO somewhere between space opera and fantasy.
>Granted I prefer hard sci-fi, but the novel’s focus is really on style over science fiction. It’s IMO somewhere between space opera and fantasy.
How I looked at Gibson's work changed completely after I read "pattern recognition" when it came out in my early '20s - It was very explicitly about style, and I went back and re-read the older stuff which I read as a child, and yeah, you could also say that neuromancer is about style and fashion. It was interesting just how much reading the later book changed how I thought about the earlier books.
Any recommendations for good hard sci-fi in the last generation or so? I'm asking because your comment strongly suggests I'd like what you like. I'm one of the very few who think that "Science Fiction and Fantasy" as a genre makes about as much sense as "Math Textbooks and Romance Novels".
I promise not to blame anyone for a recommendation that's flawed. They're all flawed. Anything where the story is based on the implications of known (well, currently accepted) science without any bogus magic is as hard as trying to figure out what will really happen in a large software project that hasn't begun yet. But what have you liked despite its flaws?
Regarding “hard” science fiction from the past 25 years, I’ve thoroughly enjoyed Stephen Baxter.
I read the first two books (Voyager and Titan) from his NASA trilogy [1]. These books are set in a near future or alternative time-line and cover inter-planetary journeys (Mars and Titan), involving the use NASA technology. Both books seem very well-researched and true-to-life.
Another book I really enjoyed was Coalescent [2]. It’s a blend of historical and science fiction: the historical part tallies with my own understanding of the late Roman Empire in Western Europe while the science part is more speculative – a human society that gradually evolves to become eusocial.
On a very different scale is, Space [3] which explores the Fermi paradox, communication between different sentient species, and the long-term survival prospects for civilisations of sentient species. Unlike the other books which have more straight-forward scientific concepts, I found some of the ideas in this book to be mind-expanding and really pushed my imagination to its limits.
From a story-telling perspective, his books are well-plotted with well-drawn, compelling characters (you really empathise with the protagonists and want to find out what happens next). I learned about a lot of diverse topics, e.g., the theories of Giordano Bruno, history of NASA projects (e.g., NERVA), the tyranny of the rocket equation, explanations of the slingshot effect, the economics of the Roman Empire, eusocial organisation and behaviour, lunar geology, Titanic meteorology, how humans could survive in a micro-gravity environment (and space in general), consequences of gamma-ray bursts, and much more.
Looking at Baxter’s Wikipedia page[3], I can see that I’ve only scratched the surface as he’s written many more books. Unfortunately, over the past decade, I’ve got out of the habit of reading novels but I really should make more of an effort.
Honestly, I have mostly given up on recent science fiction so any recommendations are welcome.
Anyway, it feels like a cop out but The Martian by Andy Weir is worth the read. The most obvious issue is the opening storm would not have done much because the atmosphere is so thin, but it is generally ok on the science side.
Agreed (unfortunately). I gave up on SF decades ago, and The Martian is the only SF book I think I've liked this century. I liked it a lot and occasionally search for others like it. In vain, it seems. I think it is an ill omen that popular culture no longer seems as excited about real science and technology, the exploration and discovery, as was the case way back in the "Space Age".
I enjoy a lot of books from Alastair Reynolds. His fiction is so and so but the science in his book is hardcore. I also liked Children of Time by Adrian Tchaikovsky (this one is two volumes).
If you don't like William Gibson... you and I have very different tastes; Neuromancer is a particularly perfect example of the sort of thing I like.
There is a certain amount of calibrating for the awards group; for my taste? the nebula is... not 100%; much like how I enjoy '80s action films rather more than Ebert. like I've never read a Pulitzer for fiction book that I didn't think was incredible, while some of the nebula books I've read were only pretty good. (I haven't read anything by Liu Cixin yet, but it sounds like my thing? I mean, it was hyped, to me at least, as genre sci-fi written from a very different cultural perspective, which is totally my thing.)
So, name five other books which are like Neuromancer and you liked them. I'll try one or two of them, if I haven't read them already. Incidentally I've read a lot of Gibson's books and Neuromancer was the one I liked the least. Probably because it was his first and not as polished as the rest. Or perhaps because by today's standards the cyber warfare he's describing sounds ludicrous. I do like his cyberpunk atmosphere though.
I've never actually read Neuromancer, but Alexander Jablokov wrote some novels similar to what I imagine Neuromancer is like. For example, Nimbus and A Deeper Sea. Near future written in the early 90s.
The writing in the English translation of The Three Body Problem is stilted and dull. Maybe it's better in Chinese? Some languages must be more expensive than others to translate well. Gave up half way through the first book; when I eventually read the synopsis on Wikipedia it sounded exciting.
I'll have to read Neuromancer again. At the time I read it I was skeptical that hacking decks and exploits would ever exist, but that was before JavaScript, internet connected critical systems, rop gadgets and nation state 0day exploit chains. The need to do scene setting for these concepts perhaps weighs on the story.
Likewise, I was pretty disappointed with The Three Body Problem after all the hype about it. Maybe it was because it was a translated work but it felt super flat for me.
I had a very hard time finding things to like in Neuromancer. It was a long time ago, but iirc the prose is pretty clunky, the characters are made of cardboard, and the plot has all the depth of a Ridley Scott movie. The world building was on point, but there's only so much 80's zeitgeist ("everything is awful, you'll be replaced piecemeal by machines, faceless megacorps rule over vast slums of pleasure junkies bathed in acid rain") that a person can channel with a straight face. Like I love me some weird dystopian cyberpunk universes, but I want that to be the starting point, not the ending point.
(Conversely, by way of making a positive contrast with something in a similar genre that , I really enjoyed A Scanner Darkly by Philip K. Dick).
To be totally clear, I think Neuromancer is probably a thoroughly enjoyable read and I'd recommend it to someone looking for what it is without hesitation. It's just pretty easy to take umbrage with.
Lol as opposed to 2019? An idiotic narcissist grifter as President, the planet on fire (southern hemisphere burning in winter), epic climate change storms, megacorp Vicodin, bikies selling crystal meth, block chain powered dark money drug markets, middle east savagery hitting new lows, internment camps in Europe and the US? China govt deploying ios 0day against their Muslim ethnic minority, and millions in camps already?
The only disappointing part is that we don't have vat grown assassins, if you want a transplant you have to go to China or Iran.
> I... personally don't understand why people even try to automate making better recommendation engines
There is a long tail of long tails: niches within niches within niches. Some of these don't have a single proven trustworthy reviewer, let alone enough that the rough edges of their opinions get sanded off by aggregation. For these ultra-niche interests, it'd still be nice to have a guide. ML can do that.
I face this problem with Netflix, Spotify and Youtube too. These algorithmic recommendations just need one improper dataset to throw everything out of the window. For weeks, my spotify is overloaded with instrumental songs. Youtube keeps on repeating the same stuff. Netflix believes that the only thing I watch is science fiction.
I call it "collapsing into the mean" - all recommendation engines (as the currently exist) will eventually corral you into the most vanilla, mass marketed set of recommendations and then fail miserably when any conflicting data is presented. We've effectively turned the web into cable TV circa 1990 - a finite set of junk food level entertainment sources that we voted for because they were "eh, good enough" and easy to find.
With as many YouTube videos that I've watched since 2005, you'd think they might recommend a video with less than 1000 views once in a while. I've found 1 new channel in the last 6 months and used the "Not Interested" option more times than I can count. And the rules are unclear. I don't want tech reviews from 5 years ago, but if I hit Not Interested, does that influence the channel, topic, age, keyword, etc recommendation frequency? I don't mind old DIY or woodworking videos, I'm subscribed to the channel, and watched the recommended video 5 years ago. (Of the current 12 YT Recommended videos, 2 are labelled Watched and only 2 are less than a year old.)
I think part of it stems from the lack of user organization features. Offer too many and you scare away users, while the people with vested interests dump money and time in to wash out any negative opinion (see Amazon reviews). Offer too few and you get poor recommendations during on-boarding/startup.
That's because the recommendation systems are optimizing for the wrong things: typically dwell time. You are highly likely to dwell longer on scifi, so heres more scifi for you!
Older recommendation systems use background and foreground timeframes to build frames of references, hence repeats.
The question to ask then is: what is the correct thing to optimize for? Dwell time is usually chosen for advertising and stickiness (you are more likely to stick with a service that plays what you like). Optimizing for novelty is difficult because the set of unknown things is much larger than the set of known things. Plus, it is risky from a business point of view.
Right. You only have four actions on Spotify as I recall (thumbs up, thumbs down, pause and fast forward). I guess some of the other apps have the larger bucket of 'who is watching/acting' based on account.
I am not sure how many actions are needed, but I can think of a few more:
It'd be nice to have an "I've already seen this" button that will mark an episode/season/show as watched so it doesn't keep coming back up. Netflix really wants me to watch Breaking Bad, but I've already seen it. I liked it; give me more like that, but not exactly that
> It'd be nice to have an "I've already seen this" button that will mark an episode/season/show as watched so it doesn't keep coming back up.
That's not just nice, that is the most obvious "feature" a normal person would design. I've stopped watching Netflix because of nonsense like this. They know I've seen it before, because I watched in their player. Also, even the tiniest, most modest application of analytics would discover that I always watch one series at the time, from start till end, or whenever I'm bored with it. In the past years, I have never ever watched a thing twice. Still, it was constantly showing me things I've already seen.
The only conclusion I can draw from this is that Netflix does not have have my best interest in mind when designing their algorithms and that they don't respect their customers. So I canceled.
Spotify has one genious feature that I use a lot: similar artists. The problem with machine learning and recommendations is that it has a few pitfalls that lead to poor results. This is on full display on Amazon which manages to recommend stuff I've already bought from them while not being able to tell the difference between hard science fiction and fantasy. The resulting recommendation bubble seems impossible to escape.
With Spotify, they do have a few features that work for me. I've discovered new artists by exploring their "Fans Also Like" feature. The nice thing about this is that they don't try to be too smart there.
Their normal recommendations suffer from the same issues that other sites have and are thrown off by the fact that my tastes are all over the place. I happily listen to sixties psychedelic rock, jazz, metal and some techno or some punk and tend to go from one to the other. Yet I'm very picky about what I listen to. Somewhere along the lines it seems to have decided I'm a middle aged guy (correct) and it consistently does not recommend me any music made this century; which is kind of frustrating if you are trying to find something new to listen to. Recommendation bubbles are a thing and escaping from them is hard.
I work around it by using the fans also like feature and using it's suggested additions to playlists. This works surprisingly well. Example based similarity search is a much simpler problem then recommendations. And it's IMHO a much more interesting feature to explore content with.
The desire for these recommendation services not to be "too smart" really resonates with me. If we can't yet give a great set of zero-effort recommendations, why don't we pull back a step and give the user a few power tools to find their own? Maybe I'm out of step with mainstream users, but I would see that as a huge improvement.
They often don't expose knobs I'd really like to tweak, either. For instance if I grew up listening to original album version of something I don't want to hear the live one. But there's only an upvote/downvote choice on Google music so what exactly am I giving the thumbs down? The artist? The song? This particular version of the song? No idea.
I absolutely agree, algorithms are terrible for recommendations. On the media database[0] my wife and I are building we're experimenting with community upvotes for suggestions. Our userbase is small so the results are inconclusive right now but hopefully time will tell if this method works.
The only good one I've ever seen was in the music business: Rdio, specifically. (Now sadly dead.) I don't know what they were doing, but their recommendations were consistently excellent at pushing me towards new artists and even new genres that I didn't expect but would up enjoying.
Back when I used to use it (about a decade ago), Pandora was good at shuffling, but their library was so small I would get the same songs over and over.
There's no reason algorithms have to be terrible, after all, most of what we do is expressible algorithmically when we give recommendations (similar interests, right level for the reader, not too advanced or too introductory, something they can use or that leads to other insights, etc.) It's just that the algorithms are generally so bad because they are optimizing for the wrong things and not any of these things that actually make for good recommendations.
Which is why the author of the article didn't find the upcoming release for the book she searched. Searching for a book we can't make money on? Pffft GTFO.
You watch one little anime with someone for high school nostalgia and suddenly Netflix thinks I'm only interested in k-pop docs, anime, and unusual shows featuring all-Asian casts... ???
Criticker has never failed me. I've been using it for years. I did hop over for a few years to a site called Jinni with really high quality UI. They closed their consumer facing services and provide their service to content providers. Criticker is still there crushing the game.
Entering "1984" yields "books similar to 1984 by Michael Dean". Entering "Nineteen Eighty-Four" gives me "Book picks similar to Nineteen Eighty-Four by Matthew Dunster". When I search for "Nineteen Eighty-Four: A Novel", the site responds with "Book picks similar to Beer in the Snooker Club by Waguih Ghali".
I'm sure Dean, Dunster, and Ghali are terrific writers but what happened to George Orwell?
"1984 orwell" finds it. Just "1984" looks like it's finding a translation or something - the cover picture clearly shows George Orwell. Maybe Dean is the translator? Still a bit odd.
Thanks for making this! I tried a bunch of searches and the recommendations are surprisingly good.
A couple of points. All my searches selected an (in my opinion) less prominent result as the top hit and I had to click the "see other results". The searches I remember doing were "Blood Meridian" (should show Cormac McCarthy as top hit and "Waiting for the Barbarians" (J. M. Coetzee).
Secondly, I'm not sure if it's sensible to show books by the same author, at least nowhere near the top.
Lastly, while I like the UI and the idea of a handwriting-type font, I find the current font a bit illegible. It's readable, but doesn't skim well.
I turned off web fonts in Firefox settings to be able to read the site easier. The recommendations seem pretty good though, so it is worth a bit of "suffering".
It takes a lot of effort to read it. I'd say that for me it's 80% unreadable. Please, at least make a button on the site to select another font if a user feels like I do.
the font is actually perfect and fits the theme of the site well, while still being easily readable. what makes it obnoxious, other than it not being some variation on fira mono or whatever?
I don't usually have an immediate positive reaction to a site, but I immediately liked yours. It's fast and simple. There is no javascript. The recommendations are reasonable.
Would you mind describing a bit of your algorithm, or is that secret info?
I applaud you for putting a site together, and I like the design look, but when I clicked on the best list of fiction from 1900-2019 Harry Potter showed up in five versions at the top of the list.
This is a hard problem because popular != good. But I would like to see data on whether recommending popular books works best. I’m sure amazon and goodreads have this data, but they are horrible at recommending.
Searched "The Glass Beads Game" got "did you mean The Everything Classical Mythology Book: Greek and Roman Gods, Goddesses, Heroes, and Monsters from Ares to Zeus by Lesley Bolton" couldnt find Hesse in additional results. Searched Inferno, could not find Dante. Kind of strange that this site misses some super important classics (well 2 of like 5 searches I tried)
Agreed. Its fixed now [1] as I added in another fuzzy search. (I was using fzf which missed some small things and I've now added Levenshtein as well to compensate).
Neat. Just a small note: On the "Best Of" page, the year selection is too narrow for my system (Chrome on Windows 10, WQHD). The two boxes only contains "20 " (twenty and some empty space); clicking/hovering one reveals those "up/down numerical selector arrows". You probably want to make them a little bit wider?
I'm really liking the recommendations on your website. I found a few good recommendations already. And like you I really struggle with finding those via the usual channels. But, that font is just awful .... :-). You do have a bit of a Harry Potter infestation on your best fiction list. It appears multiple times there. Same with game of thrones related stuff.
Out of curiousity, where do you source your data from?
Awesome! Indeed feels like a better selection of recommendations
The only tweak I’d make? Link to UK (etc, but UK for me) Amazon too! Mainly because I’m lazy but also you could then set up a UK affiliate tag. I buy a lot of books, get your slice!
your site is useful and has good recommendations. Thanks for the link. I tried looking at books with "Writing" category and some of the recommendations like "On Writing by Stephen King" are really good and what I would expect. Though I feel that you should work on your priority of order when showing recommended books. For example: when I search for books similar to "The Handmaid's Tale", I don't expect to see "Cat's eye" as a top recommendation. Something to improve upon further.
The Kindle ads are worse. I don't mind at all opening up my Kindle and seeing a book recommendation ... if its at LEAST in the genres I read. You would think Amazon would be on top of it.
How can you recommend books based on only one observation ? Wouldn't it make more sense to ask many books the user liked in order to make a better "user hyperplane" ?
By the way, can I ask how you calculate the recommendations? Your explanation on the site is that it just uses the genre, but that cannot possibly be the whole answer?
I agree. Collaborative filtering should succeed when you have sufficient of data, but we don't always have sufficient data. I think books are a tough challenge: the number of books is relatively large (vs. e.g. the number of restaurants on Yelp), the number of books people read is relatively small (vs. e.g. how many things they buy). Plus, people's tastes in them are relatively particular, and they're a big time commitment, so the bar for what counts as a "good" recommendation is high.
I think in these cases you can do better by bringing in some content-based filtering. I made an experimental book recommender using only story trope tags and I thought the results were already better than what I was getting elsewhere. It's still up at https://bookslikethis.herokuapp.com/ (but it basically only has sci-fi/fantasy titles).
Recommendations seem good for what I checked. One bit of weirdness though, searching for Gödel, Escher, Bach [1] gets me a book by "Agnes F. Vandome", that on googling is some sort of fake book of compiled wikipedia articles, and I need to click on the alternatives to find Hofstadter's book. So it looks like the search system can be successfully spammed with fake titles for reasonably notable books.
I had the same experience with GR and also Amazon.com which constantly peddles the vampire romance books when I am looking for recommendations for horror/fantasy. Both Amazon and GR strategy make sense because best-selling books sell the best, so they should recommend them to increase profits. However, it does suck being a reader looking for new book suggestions.
I've spent a good deal of time making my own book recommendation algorithm which has been working well for me for the last two years. [1] Through it I've discovered old authors I didn't know (Ted Chiang, Clive Barker) and new authors which I wouldn't have noticed (Scott Hawkins, China Mieville). Of course, it always helps to get recommendations from friends with similar tastes, too. :)
[1]: https://nowwhatdoiread.com