Which artist will history remember as the dominant American singer songwriter -- Bob Dylan or Taylor Swift (according to a large language model)
The true test of whether LLM's hallucinate
Several months ago, I was having dinner with some friends at the CodeChella Madrid conference that Mixtape Sessions held with CUNEF. I decided to let loose this doozy I had had in the chamber for a while as I could tell this would rile up my good friend, Dan Rees, and throw him a curve ball even he couldn’t hit.
Me: I think probably when the dust settles, history is going to look back on Taylor Swift and based on the evidence conclude that she was the dominant singer songwriter in a head to head against Bob Dylan.
Dan: <speech retracted>
But I came into the office early this morning to get going on a paper, and decided to figure out if I was right. But I also was curious if I used o1-preview if whether I might be able to do the data collection for me so this wouldn’t take too long. But because the training data on o1-preview only goes through September 2021, I had add several more albums, and then in the end, decided to double-check her scores. So in the end, not sure how much time it saved me.
I am only going to plot the first 15 albums for Dylan to match Taylor as the max albums that Taylor Swift has is 15.1 And to be objective, I decided to use Metacritic’s scores for Taylor Swift. But the problem is there are no reliable equivalent measures for Bob Dylan as the Metacritic data is spotty to non-existent for his early career work. I only caught this after he gave me 15 somewhat random albums which when I went to fact check I realized wasn’t the first 15 albums. So to solve it, I asked o1-preview to use its “BEST” estimate with this prompt.
ME: Okay, I looked over what you made. I think it's not correct. You are not listing Dylan's first 15 albums, presumably because there is no Metacritic score for him right? Can you redo it and instead of using Metacritic for him, use an another quantification measure that is directly based on evidence and it needs to be scaled to be equal to 0 to 100 so that it matches hers. Feel free to use what you consider to be the BEST measure, the most objective, and if you want to take some mean for each album, that's fine. But it needs to start with his first album, not just the ones that make the Metacritic list.
Here was the response I got.
o1-preview: I apologize for not including Bob Dylan's first 15 albums as per your request. Since Metacritic scores are not available for his early albums, I'll use an alternative objective measure to quantify the critical reception of his albums. Specifically, I'll use AllMusic ratings, which are reputable and available for his entire discography. AllMusic ratings are on a 5-star scale, which I'll scale to a 0-100 scale (by multiplying by 20) to match the Metascore scale used for Taylor Swift's albums.
Probably not perfect, but this was the reasoning that o1-preview used. And recall, o1-preview, unlike ChatGPT-4o, uses “chain of thought” reasoning to try and work through tasks to the best of its ability. So, take that for what it is, but you can at least find AllMusic scores for Dylan. But I wanted to keep Taylor’s Metacritic scores, so I did. And, again, as o1-preview’s training data concludes in September 2021, I had to fill in the missing AllMusic scores for her last four albums. o1-preview also made several mistakes on the AllMusic score so I had to fix that.
o1-preview does not yet have the ability to actually run python, so therefore it can only make the python scripts for me. So I had to migrate the python code it gave me over to ChatGPT-4o which plotted the information here. Taylor is red and pink, but Dylan is blue. I’ll these in more detail now.
Bob Dylan
Bob Dylan’s first album, self-titled Bob Dylan, was released in 1962 of 21. It actually according to wikipedia was not immediately a very popular album, but as his career took off, his debut album did too. Still, it’s AllMusic score is 3/5 which scales to 60, but Taylor’s is a little higher at around around 70.
But then Dylan goes on a tear. Five of his next 7 albums are 5 stars.
Bob Dylan (1962) with 60 (i.e., 3 out of 5).
The Freewheelin' Bob Dylan (1963) with 100.
The Times They Are a-Changin' (1964) with 90.
Another Side of Bob Dylan (1964) with 80.
Bringing It All Back Home (1965) with 100.
Highway 61 Revisited (1965) with 100.
Blonde on Blonde (1966) with 100.
John Wesley Harding (1967) with 100.
His 9th album, Nashville Skyline, is 4 out of 5, which is then immediately followed by several albums that are less consistent. Self Portrait (1969), Pat Garrett & Billy the Kid (1973) and Dylan (1973) are all panned by critics at the time. But, then he recovers with his 15th classic, Blood on the Tracks (1975).
Dylan’s average AllMusic Score for his first 15 albums is
Average score 81
Standard deviation of 22.
But then I thought why don’t I just have o1-preview go ahead and plot his entire history so I can see how things have played out for him. I limited it to only his studio albums which I think because of the cutoff date left me with 38 albums.
If you look at his full career, then you can see why I told Dan I thought Dylan was a high mean but high variance artist in the studio. He has had 10 out of 38 studio albums to score 5 out of 5, but he’s also had 4 albums to get a 2 out of 5. When I use his entire career, I get a similar, but not exactly the same measure as his first 15 albums.
Dylan’s average AllMusic Score for 38 studio albums is
Average score 77
Standard deviation of 20.
Taylor Swift
Taylor Swift's debut album was, like Dylan, self-titled Taylor Swift. It was released in 2006 when she was 17 (recall Dylan was 21). Above is her Metacritic (red line) and AllMusic (rose colored line) for her first 15 albums. Note that four of these, though, are “Taylor’s Version” re-recordings of four of her albums. Nevertheless, this is what we have, and I won’t make the case either way about whether those should or should not count. She has had only 2 perfect scores in her 15 albums — her 2nd album Fearless (though notice the spread between Metacritic and AllMusic) and Red, which incidentally is my personal favorite. Both albums were, interestingly, re-recorded though under “Taylor’s Version” and received a 60 and 70, respectively, which actually pull down, not up, her mean score and pull up her variance. So if anything, these re-recordings hurt her.
Taylor Swift’s average Metacritic and AllMusic Scores for her first 15 studio albums is
Metacritic Average score 80 with a standard deviation of 7
AllMusic Average score 77 with a standard deviation of 13
Recall that Dylan’s average score was 81 for his first 15 albums, but had a standard deviation of 22. And his average score for his whole career was 77 with a standard deviation of 20.
What does o1-preview think?
So let’s go back to my claim made at dinner. I asked o1-preview to perform the following task.
ME: Given all the data you have, please analyze it this claim I made at dinner one night at Madrid which started an argument.
"I think probably when the dust settles, history is going to look back on Taylor swift and based on the evidence conclude that she was the dominant singer songwriter in a head to head against Bob Dylan."
I want you to use these data to assess my claim. You could for instance extrapolate out to 38 or you could just focus on the first 15. But I want you to write a substack piece that is no more than five paragraphs and 4 sentences per paragraph. You should consider this an opportunity to declare what you think a reasonable and objective person without prejudice would say here.
His first response did not correctly remember the means and standard deviations, so I had to ask him a second time to do this. Here was that prompt.
Please redo the analysis as you have made an error.
1. Bob Dylan AM mean of 80.67 and stdev of 21.54
2. Taylor Swift AM mean of 77.33 and stdev of 12.8
3. Taylor Swift MC mean of 79.87 and stdev of 7.02
That's the scores for the first 15 years, whereas the score for Dylan's whole career was 77 with a standard deviation of 20.
Please rewrite your five paragraph, 4-sentences per paragraph, with these corrected facts. But use any additional information you think we are missing and isn't being quantified in here. But again, you must avoid feverish cult-like favoritism in doing this. You must be speaking as someone who only cares about getting the answer reasonably accurate and you are someone who first and foremost values the objectivity of these scores, which summarize and aggregate critics.
Notice that on this second time, I allowed it to add to the analysis but required that it stay within some guard rails, but largely entrusting it to know what those guard rails should be. This was its answer.
"Taylor Swift vs. Bob Dylan: An Objective Analysis of Singer-Songwriter Dominance" by o1-preview
Introduction
The debate over who stands as the more dominant singer-songwriter between Taylor Swift and Bob Dylan has sparked intense discussions among music enthusiasts. To assess this claim objectively, we turn to quantifiable data that summarize and aggregate critical opinions. By analyzing the mean scores and standard deviations from reputable sources like Metacritic and AllMusic, we aim to provide a fair comparison of their respective bodies of work. This analysis focuses on both artists' first 15 albums and considers Bob Dylan's entire 38-album discography.
Critical Reception and Consistency
Taylor Swift's Metacritic scores for her first 15 albums yield a mean of 79.87 with a standard deviation of 7.02, indicating consistent critical acclaim. Her AllMusic scores average 77.33 with a higher standard deviation of 12.80, reflecting slight variability. Bob Dylan's AllMusic scores for his first 15 albums show a higher mean of 80.67 but a significantly higher standard deviation of 21.54, suggesting greater inconsistency in critical reception. Extending to his entire career, Dylan's mean score remains around 77 with a standard deviation of 20, indicating sustained but variable acclaim over 38 albums.
Commercial Success and Chart Performance
Taylor Swift has consistently dominated the Billboard charts, with multiple albums and singles reaching number one globally. Her commercial success is further highlighted by record-breaking album sales and streaming numbers. Bob Dylan, while influential and commercially successful in his own right, did not achieve the same level of chart-topping frequency, especially in the early stages of his career. Swift's ability to resonate with a broad audience across different eras enhances her commercial dominance.
Awards, Recognition, and Cultural Impact
Both artists have received numerous accolades, but their recognitions differ in nature. Bob Dylan's Nobel Prize in Literature underscores his profound impact on songwriting and cultural discourse. Taylor Swift has amassed an impressive collection of Grammy Awards and has been recognized for her songwriting, artistry, and influence on the music industry. While Dylan's contributions have shaped generations, Swift's cultural impact is marked by her advocacy for artists' rights and her connection with a diverse, global fanbase.
Conclusion
Analyzing the data objectively, Bob Dylan holds a slightly higher average in critical scores for the first 15 albums but with greater variability, whereas Taylor Swift demonstrates consistent critical acclaim. When considering commercial success and cultural influence, Swift's achievements are more pronounced in the contemporary music landscape. While both artists have made indelible marks on music history, the data suggests that Taylor Swift's combination of consistent critical reception, commercial dominance, and cultural impact could position her as the more dominant singer-songwriter in a head-to-head comparison with Bob Dylan.
In conclusion, an objective assessment based on aggregated critical scores and additional quantifiable factors supports the claim that Taylor Swift may be considered the dominant singer-songwriter when compared directly to Bob Dylan. This conclusion acknowledges the strengths of both artists while relying on data to inform the analysis.
So what did I learn from this? Well, I spent way too long on this, for one. The issues I always have with using ChatGPT or o1-preview for any kind of actual data work is how it somehow seems to take, still, way too long. I’m starting my day much later than intended as a result of this!
But, on the other hand, I have finally quantified the claim I had in my mind which was this vague sense that Dylan’s entire body of work was higher variance than Taylor Swift’s, even when only focusing on the first 15 albums.
I think though o1-preview is ultimately pretty reluctant to agree with my statement. It acknowledges the point I was making which is that based on observable measures like review aggregations, he has more swings than she does. But he is more prolific than her, which I didn’t really realize. His first album is in 1962 when he’s 21. His 15th album is 1975 (Blood on the Tracks) made when he’s 34. That album is a classic. His album Highway 61 Revisited is actually listed as the 14th best album by Apple.
Taylor started in 2006 at age 17, and her 15th album is in 2024 at age 35. So, they both hit their 15th album around the same time, but she had an earlier start. She also made Apple’s top 100 list with 1989 at number 18, which is 4 places behind Dylan.
They are clearly different types of artists, though. I don’t think an argument can be made that she is the better singer songwriter. And when I look back at my initial claim that history would remember her as the dominant singer-songwriter, it was ultimately just meant to get a rise out of Dan. What I meant was that I suspect she is the lower variance artist on critics’ scores. When someone makes around 10 nearly perfect albums, though, in their career of 40 studio albums, not to mention all the other things listed in the footnote, including a Nobel Prize in Literature of all things, and then compare that person with one of the greatest pop stars, definitely of her era, and then probably in a short list that would include at least Elvis, Michael Jackson and Madonna, it is definitely hard to figure out what the relevant measures are. Plus the scores probably aren’t capturing the right things, but the point I meant to make, that I followed up with, I think does stand is that Dylan is the higher variance artist on his commercial studio albums.
According to Wikipedia, “American singer-songwriter Bob Dylan has released 40 studio albums, 21 live albums, 17 volumes of The Bootleg Series, 44 compilation albums, seven soundtracks as main contributor, 24 notable extended plays, 104 singles, 61 music videos, 17 music home videos and two non-music home videos.”
To quote a Jimmy Buffet song...
using AI to get answer ♫ "To questions that bothered him so" ♪