Saturday, May 11, 2013

Steak Tasting

New York Strip Steak Tasting
Sat 05/11/2013, 03:00p-09:00p


This has been a long time coming. Several of my friends and I had been talking about doing a steak tasting for a while now, so this past weekend we finally muscled up the gumption to actually go through with it. We purchased nine NY strip loins from a wide variety of vendors ranging from ultra high-end mail order companies to local butcher shops to mainstream chain retailers. The steaks are summarized in the table below, sorted in descending order of price:

Steak Name Price/lb Price Weight Thickness Aging Grade Notes
Flannery California Reserve New York Steak $55.78/lb $62.75 18oz (advertised) 1 3/4" (1 1/2" advertised) 4-week dry aged Prime Additional shipping charges may apply.
Pat LaFrieda 50 Day Dry-Aged USDA Prime Black Angus Bone-In NY Strip Steak $48.89/lb $55.00 18oz (advertised) 2" (1 1/2" advertised) 50-day dry aged Prime No antibiotics, hormones, or growth promoting drugs. Fed grass/hay and finished on corn. De-boned prior to shipment. $20 shipping charge not included in price.
Flannery Midwestern New York Steak $43.11/lb $48.50 18oz (advertised) 1 7/8" (1 1/2" advertised) 4-week dry aged Prime Additional shipping charges may apply.
Lindy & Grundy NY Strip 6 Week Aged $41.00/lb $41.82 16.32oz 1 5/8" (1 1/2" requested) 6-week dry aged Ungraded Organic. No antibiotics, hormones, or pesticides. 100% grass fed and grass finished. De-boned by request at point of purchase.
Beef Palace Extra Dry-Aged Prime NY Strip $25.99/lb $19.70 12oz 1 5/8" (1 1/2" requested) 3-week dry aged Prime  
Whole Foods Dry Aged Porterhouse Steak $21.99/lb $34.96 25.44oz 1 1/2" 4-week dry aged Prime Vegetarian diet with no antibiotics or added growth hormones. Was purchased as a trimmed and de-boned porterhouse.
Costco Kirkland Signature Prime New York Steak $11.99/lb $9.07 12.11oz 1 1/4" Unknown Prime Blade tenderized.
Ralphs Select Boneless New York Steak $10.99/lb $13.74 20oz 1 7/8" (1 1/2" requested) Yes, wet aged Select Custom cut and not pre-packaged.
Walmart Choice New York Strip Steak $10.48/lb $8.54 13.04oz 1 1/8" Yes Choice  


Steak Lineup Decryption Keys
In an attempt to eliminate bias, we tried to do the tasting as blind as possible. As such, what I did was remove the steaks from their packaging, lay them out bare on plates, and assign a letter to each one (as seen on the piece of paper to the left). The person actually cooking the meat then came in and, not knowing my letter assignment and thus which steak was which, randomly assigned a number to each letter (the paper on the right), keeping the mapping away from me. In effect, we had a two-part "encryption" method wherein no one person would know both parts of the "key" necessary to relate the name of the steak with its assigned number. The two "keys" were only put together at the end of the tasting.

Salt and Pepper Only Seasoned Steak
With that out of the way, another member of the tasting party seasoned all sides of each steak with a mixture of ground pink Himalayan salt and Tellicherry peppercorn.

Vacuum Sealer Sealed Steak
The steaks were then sealed in plastic bags using a FoodSaver V3240 vacuum sealer.

Pappy Van Winkle's 20yr Family Reserve Kentucky Straight Bourbon Whiskey
During the prep process, we sampled the Pappy Van Winkle's 20yr Family Reserve Kentucky Straight Bourbon Whiskey that one of the tasters brought. Good stuff!

Sous Vide Steak Cast Iron Finish
With all the meat prepped and ready to go, we plopped the bags into a PolyScience immersion circulator. The steaks were cooked sous vide at 123°F for at least two hours, then finished on cast iron (one minute on the top and bottom, then 30 seconds for the sides) with butter immediately prior to serving.

Slicing Steak Bite Size Pieces
The steaks were selected in random order, finished, sliced, photographed, and then sliced some more into bite-size pieces.

Steak Scorecard
The meat was then brought to the table for our nine tasters to sample, identified merely by a number. Tasters were provided Shun knives and score sheets, which were collected at the end of the evening and tabulated. Along with the steak, we were also served mashed potatoes and a simple salad.

Scholium Project Gardens of Babylon
In a nod toward consistency, we all drank the same wine throughout the tasting, the Scholium Project Gardens of Babylon, a bright, lively Petite Sirah-based California blend that stood up well to the meat without overshadowing it. Water was also provided, in addition to cola for the teetotaler in the group.


The steaks are shown below in the order that they were tasted. I'll present my personal thoughts on each before delving into the aggregate results.

Flannery California Reserve New York Steak Flannery California Reserve New York Steak
#10 - Flannery California Reserve New York Steak
We got started with what was easily the most expensive (and most beautiful) steak of the group. What the price bought you was some pretty incredible marbling, in fact probably the most marbling I'd ever seen in a non-wagyu-style steak. The result of this was a super intense, super fatty flavor that was delicious, but at the same time, almost too much. The issue here, though, was the texture, which was quite chewy at times. This could've greatly benefitted from some more cooking.

Pat LaFrieda 50 Day Dry-Aged USDA Prime Black Angus Bone-In NY Strip Steak Pat LaFrieda 50 Day Dry-Aged USDA Prime Black Angus Bone-In NY Strip Steak
#5 - Pat LaFrieda 50 Day Dry-Aged USDA Prime Black Angus Bone-In NY Strip Steak
Next was the second priciest steak, from famed New York butcher Pat LaFrieda. This one showed off an almost foie gras-esque richness and lushness to it, and just oozed fat upon mastication. Unfortunately, it also suffered from the textural problems witnessed in the Flannery above, but was overall more balanced and easier-eating.

Costco Kirkland Signature Prime New York Steak Costco Kirkland Signature Prime New York Steak
#1 - Costco Kirkland Signature Prime New York Steak
The Costco steak shined in the area of texture, especially compared to the two preceding cuts. It was obviously much more restrained in character, with a slight nuttiness and a nice crust, and was something that I'd have no problem eating a large portion of.

Beef Palace Extra Dry-Aged Prime NY Strip Beef Palace Extra Dry-Aged Prime NY Strip
#9 - Beef Palace Extra Dry-Aged Prime NY Strip
Beef Palace is a well-regarded butcher located in sunny Huntington Beach, CA. I'd heard some great things about the place, and its product did not disappoint. I actually found it quite similar to the Kirkland steak, and also rated it highly.

Walmart Choice New York Strip Steak Walmart Choice New York Strip Steak
#8 - Walmart Choice New York Strip Steak
The Walmart steak pleasantly surprised most of us, myself included, despite being the thinnest of the bunch. It had a great sear and char, and was balanced, mild, and non-offensive in presentation. There wasn't a huge amount of flavor here, but I really didn't mind eating it.

Whole Foods Dry Aged Porterhouse Steak Whole Foods Dry Aged Porterhouse Steak
#4 - Whole Foods Dry Aged Porterhouse Steak
I'd had Whole Foods steaks before, and they'd always been solid. This was no exception, displaying a very well-integrated fattiness and a delectable earthiness on the close.

Flannery Midwestern New York Steak Flannery Midwestern New York Steak
#3 - Flannery Midwestern New York Steak
For me, the second Flannery entry also suffered from the same problems we saw in the first, which, unfortunately, was likely a byproduct of our methodology rather than an inherent flaw in the meat. That being said, the steak was certainly flavorful, but parts of it seemed strangely bland at times.

Lindy & Grundy NY Strip 6 Week Aged Lindy & Grundy NY Strip 6 Week Aged
#7 - Lindy & Grundy NY Strip 6 Week Aged
Upon biting into the L&G, I knew instantly that this was grass-fed. The eating experience is just that different. The flavor here was intense to say the least, with a grassy, almost seaweed-like taste that's certainly not for everyone (though I'm ok with it). Texturally, I found this surprisingly pleasing despite the leanness of the meat.

Ralphs Select Boneless New York Steak Ralphs Select Boneless New York Steak
#2 - Ralphs Select Boneless New York Steak
We ended with a USDA Select cut from Ralphs, which is probably representative of your typical supermarket steak. It was definitely on the lean side, though I thought it was surprisingly juicy. Taste-wise, I wasn't in love, deeming the meat minerally and austere, with a nutty tinge. It was my least favorite of the bunch.

Now that we have my thoughts out of the way, we'll take a look at the aggregate data for the entire group. In the table below, I show the average (mean) values for all the steaks across each category, as well as the resultant rankings. I also have some select comments from the other tasters, some of which were quite eye-opening.

Steak Name Taste Texture Looks Total Overall Rank Taste Rank Texture Rank Other Tasters' Comments
Flannery California Reserve New York Steak 11.1 6.4 4.1 21.7 6 4 9 Rubbery. Meaty. Wagyu like. Very chewy.
Pat LaFrieda 50 Day Dry-Aged USDA Prime Black Angus Bone-In NY Strip Steak 12.3 7.3 3.9 23.5 3 2 8 Beautiful beefy flavor. Firm. Tastes like steak.
Flannery Midwestern New York Steak 12.1 8.2 4.2 24.5 2 3 2 Buttery but a touch gritty. Lil' earthy.
Lindy & Grundy NY Strip 6 Week Aged 7.1 7.4 3.6 18.1 9 9 7 Super grassy. Intensely earthy. Livery. Steely, seaweed. Eww.
Beef Palace Extra Dry-Aged Prime NY Strip 10.8 7.8 3.9 22.4 4 5 4 Non offensive. Yummy.
Whole Foods Dry Aged NY Strip 12.3 8.6 4.0 24.9 1 1 1 Fatty. Not intense. Super good.
Costco Kirkland Signature Prime New York Steak 10.1 8.1 3.3 21.6 7 7 3 Grainy. Meh.
Ralphs Select Boneless New York Steak 10.2 7.5 4.0 21.7 5 6 5 Ultra nutty. Juicy. Super beefy. Steely. Pretty good.
Walmart Choice New York Strip Steak 8.8 7.4 3.4 19.6 8 8 6 Not much flavor. Texture is good. Tastes of butter. Not as yummy.

Though the rankings are obviously important, they're only part of the story, and I think it's also worthwhile to take a look at the variability in the scores:

Steak Name Range (High - Low Scores) Standard Deviation Relative Standard Deviation
Taste Texture Looks Total Taste Texture Looks Total Taste Texture Looks Total
Flannery California Reserve New York Steak 2 5 2 5 0.8 1.4 0.9 1.8 7% 22% 23% 8%
Pat LaFrieda 50 Day Dry-Aged USDA Prime Black Angus Bone-In NY Strip Steak 4 2.5 2 5 1.2 1.0 0.9 1.7 10% 14% 24% 7%
Flannery Midwestern New York Steak 5 4 2 11 2.0 1.7 1.0 4.2 17% 21% 23% 17%
Lindy & Grundy NY Strip 6 Week Aged 10 3 1 12 3.1 1.1 0.5 3.8 44% 14% 15% 21%
Beef Palace Extra Dry-Aged Prime NY Strip 8 3 2 11 2.6 1.1 0.8 3.7 24% 14% 20% 17%
Whole Foods Dry Aged NY Strip 3 3 2 8 1.3 1.1 0.9 2.8 11% 13% 22% 11%
Costco Kirkland Signature Prime New York Steak 5 4 1 7 1.6 1.4 0.5 2.5 16% 17% 15% 11%
Ralphs Select Boneless New York Steak 6 3 2 9 2.2 1.1 0.9 3.4 21% 14% 22% 16%
Walmart Choice New York Strip Steak 4 5 3 8 1.4 1.7 0.9 2.8 16% 23% 26% 14%

  • Whole Foods snags the win here, coming in first overall, as well as for the subratings of taste and texture. I'm positing that their secret for success was that the steak was generally very enjoyable to all the tasters, almost serving as a sort of "middle ground" with its tender, non-gristle-y consistency, moderate fattiness, and pleasing, though not overbearing flavor. There was nothing to dislike about the steak, and indeed, all of the tasters rated it at least moderately high.
  • The correlation between deliciousness and price wasn't as strong as we were expecting. Even the least expensive, commodity steaks of the bunch were serviceable. Hell, the Ralphs Select came in in fifth place (though personally, it was my least favorite of the bunch).
  • As far as appearance went, all the steaks looked relatively similar when cooked, so next time, I'd probably skip the "Looks" subrating. Going even further, perhaps we should eliminate subratings altogether and just go with an overall score. Keep it simple right?
  • Looking at the variability data, we see that people tended to be in agreement about the LaFrieda and the Flannery California Reserve, while the Lindy & Grundy was undeniably polarizing, as was the Walmart steak.
  • We aimed for a consistent 1.5" thickness across all the steaks, but this was difficult to enforce in practice. We found that the NY strips were, in every case, thicker than specified.
  • In certain cases, there was a large disparity between a steak's taste and texture scores. For example, the Flannery California Reserve scored well on taste, but was dead last for texture; the LaFrieda was even starker in that regard. On the other hand, the Costco steak fared far better in texture than taste.
  • I'm sure that part of the inconsistencies in the bullet point above was due to our cooking methods. The super-premium steaks--the Flannery and LaFrieda--were all rather fatty, and thus could've benefitted from a higher temperature setting on the immersion circulator, something closer to 130°F. This would've done a better job of cooking and integrating the marbling, which would likely get rid of complaints that the steaks were overly firm and chewy. Unfortunately, doing so would probably overcook many of the other steaks. In any case, I do believe that the LaFrieda and Flannery were shortchanged somewhat, and could've done much better.
  • As the sole grass-fed steak in the lot, the Lindy & Grundy bore the brunt of the tasters' unfamiliarity with the meat. Grass-fed steaks account for only a small percentage of the beef market, and most people simply have a hard time accepting the significantly different flavor profile of the meat, which I'm sure was only amplified by the six-week dry aging we had here. It's a leaner, healthier type of beef that's also better for the animals and the environment, but is definitely an acquired taste as we've seen here. Just look at the huge 44% RSD value for its taste rating; some people really didn't like it. I'm not surprised at the result, but for next time, perhaps an unaged grass-fed cut would fare better.
We had a blast holding this steak tasting, and I'm glad I enforced a bit of rigor to the process, which made for some enlightening results. I'm satisfied with how things turned out, but I think we definitely had some lessons learned as well about how we'd do things differently next time (and I'm pretty sure there will be a next time, I'm thinking with rib eyes). For Steak Tasting 2.0, what should we change? Which other steak purveyors should be included in the comparison?


Blogger Epicuryan said...

I think for 2.0 we should do it at your house.

Tuesday, May 14, 2013 6:53:00 AM  
Anonymous Anonymous said...

Very cool idea/execution, and I'm glad to see that some of the "lesser" steak options hold up well. As someone without a sous vide machine what cooking method(s) would you recommend? Pan sear and finish in an oven?

Tuesday, May 14, 2013 9:51:00 AM  
Anonymous dudetb3 said...

have you ever considered going to Le Tub in florida. Easily served the best chili, Burger, and key lime pie i have ever had. Along with that you are eating right on the coast line.

Tuesday, May 14, 2013 10:01:00 AM  
Blogger Charlie Fu said...

LOL at Ryan's comment :). Your kitchen looked like a butter bomb exploded on the stove top.

Anonymous: Depends how you like your steaks. If your steak is thick enough (1.5+) I'd do a low roast in the oven at 250F up to temp then a sear. That would be similar to a sous vide method, but not quite as accurate. I do this for large 2inch + steaks when I'm cooking more than a few at once.

If you're looking at thinner steaks, i'm a big proponent of cast iron pan with flipping every 30 seconds in butter and other herbal aromatics. Yields a very evenly cooked steak.

Kevin: Now I know why you were taking so long upstairs, that is an impressive chart my friend!

Tuesday, May 14, 2013 1:04:00 PM  
Blogger sygyzy said...

While I am a fan of rare steaks, I don't think you'll find many people, even Modernist Cuisine chefs, that would advise cooking at 121F. Not because it's dangerous (I don't really care) but because it seems a tad too rare for me. It explains why both you and Epicuryan complained that some of the fattier (marbled) cuts were too chewy. Fat needs heat to render ...

That point aside, WHAT AN AWESOME thing to do! I wish I could do such a tasting but alas funds are limited. What I got from this though is a guide to figure out which steaks to buy when I want to treat someone to a nice meal in. What'd be cool is if you created some sort of index for Rating vs Price ratio. What's the best bang for your buck steak?

Tuesday, May 14, 2013 1:22:00 PM  
Anonymous Anonymous said...

@sygyzy: completely agree with your comment "Fat needs heat to render"... there's a reason why most of, if not all, the finer steak houses cook at a super high temp or over open flame. that said, not everyone has a commercial grade oven in their house... i did a similar tasting once but the steaks were BBQ'd over red oak and we had about half as many... it'd be pretty hard to keep track of 2x as many steaks on a grill!

Tuesday, May 14, 2013 1:48:00 PM  
Blogger normchow said...

a fantastic read!

Tuesday, May 14, 2013 4:46:00 PM  
Anonymous JustinM said...

I agree another amazing review Kev! I spend all day looking at meat online but your pics make my mouth water! Might organize a meat tasting party with the guys soon.

Tuesday, May 14, 2013 8:16:00 PM  
Anonymous pdaddy916 said...

Awesome post and seemed to be very well executed. You should look into Myer's Beef out of Montana for steak tasting 2.0

Tuesday, May 14, 2013 8:36:00 PM  
Blogger Epicuryan said...

Charlie... you don't know the half of it my house still reeks of butter and meat

Tuesday, May 14, 2013 8:38:00 PM  
Anonymous Anonymous said...

I can't imagine putting a good steak inside a plastic bag and then letting it sit in tepid water. Then you sort of fry it?

It's California. Go outside and grill the steaks over coals.

Wednesday, May 15, 2013 2:51:00 AM  
Blogger Unknown said...

I love a good steak grilled outside on some real charcoal, since it gives the steak so much character and soul. However for a taste test, where you really want a consistent method, sous vide is the best method. Grilling is too inconsistent. It's also important to note that these prime cuts are essentially developed through time as the best cuts for direct cooking. Sous vide evens the field because you could make a cheap cut of beef taste like a prime cut.

For me, the magic temperature is 126F if I sous vide. I don't have an immersion circulator at home, just a temp regulator and an old slow cooker, so I too set it to 121F because it could run up to 10 degrees hotter. I've done it with really cheap, thin steaks, and it doesn't over cook the steak. However if you're worried about it being overly cooked, you could drop the bag in iced water for 1 minute. That should cool the surface temp enough so that you could sear it and not over cook the interior.

You could also try to sous vide longer, but you risk producing a rare fork tender steak without any bite or chew.

It's probably best to push for a higher temperature setting. Maybe do trials with the cheapest steaks you could find, and then do another taste test with the expensive stuff.

Wednesday, May 15, 2013 11:51:00 AM  
Anonymous Josiah said...

For next time I suggest two local butcher shops that I like better than L&G (and I too hate the super-aged grass fed beef there):

- McCall's Meat and Fish Company in Los Feliz (my go-to butcher shop when getting the good stuff)
- Huntington Meats in the Farmer's Market (80% Grass fed / 20% Grain fed - Prime Harris Ranch Beef)

Wednesday, May 15, 2013 12:23:00 PM  
Blogger Charlie Fu said...

for everyone who says "grill grill grill", check this link out

Wednesday, May 15, 2013 5:47:00 PM  
Blogger kevinEats said...

Ryan: Or Charlie's.

Anon: I'd probably go pan searing, and actually proposed that method in lieu of sous viding when we were still in the planning stages.

dudetb3: Not sure what that had to do with this post...

Charlie: Thanks for chiming in there. And yes, I was working furiously in Excel up there while you guys were complaining about how long it was taking!

Sygyzy: Agree about the temperature--a lesson learned for next time. Perhaps we should try to get steaks with comparable marbling. Here's the ranking of average overall score over price per pound:

Ralphs 21.7 / 10.99 = 1.98
Walmart 19.6 / 10.48 = 1.87
Costco 21.6 / 11.99 = 1.80
Whole Foods 24.9 / 21.99 = 1.13
Beef Palace 22.4 / 25.99 = 0.86
Flannery MW 24.5 / 43.11 = 0.57
LaFrieda 23.5 / 48.89 = 0.48
Lindy & Grundy 18.1 / 41.00 = 0.44
Flannery CA 21.7 / 55.78 = 0.39

Anon: Agree as well. I'd be curious about grilling as well, though I don't think any of us actually had a grill.

Norm: Thanks. At first, I was hesitant about posting this article, but I'm glad I did.

Justin: If you do, are you gonna blog it? ;)

pdaddy916: Thanks for the recommendation! They don't seem to be available locally, so it'll have to be mail order.

Ryan: It wasn't that bad when I left, but maybe that's because I got used to the smell.

Anon: Though I do enjoy coal-fired steak, the concern there is consistency across the different cuts.

Barack: Great comment--I think you really hit the nail on the head. As for the temp, 126F sounds reasonable, and I sorta wish we'd done that here. I wonder how that would've altered the results.

Josiah: Will have to keep those in mind. Heard good things about 'em!

Charlie: So we pretty much got it right, save for the higher temperature and blowtorch.

Thursday, May 16, 2013 2:30:00 AM  
Blogger Unknown said...

If you really want to go mondernist with the steaks you could cryofry the steak. Basically sous vide the steak at 57C (134F), followed by a 30 second dunk in liquid nitrogen, and then a 35 second deep frying. That would probably be the most consistent cooking method.

Friday, May 17, 2013 1:55:00 PM  
Anonymous Casey said...

I'm curious how Trader Joe's compares. If you do another taste test :)

Sunday, May 19, 2013 2:00:00 AM  
Blogger thngotiatr said...


I don't think Trader Joe's would be a good candidate as they don't have their own butcher shop in any of their stores. Everything is ordered and delivered off site so there is no consistency as far as who the specific supplier is. This is in no means a knock on the quality, it's just there is too much inconsistencies.

Sunday, May 19, 2013 8:59:00 PM  
Anonymous Sgee said...

You guys were over engineering this taste test.

Should have just pan seared over a smoking cast iron pan or over a charcoal grill. Just get this cheap but effective grill|1&pl=1&currentURL=%3FNs%3Dp_product_qty_sales_dollar%7C1&facetInfo=Char-Griller

Great idea though

Tuesday, May 21, 2013 8:27:00 PM  
Anonymous Sgee said...

Add Harvey Gus's to the list next time

Tuesday, May 21, 2013 8:29:00 PM  
Blogger kevinEats said...

Barack: I don't think we have the gumption for that lol.

Casey: I'll keep Joe's in mind, also Bristol Farms.

Vinh: Thanks, didn't know that. What about Bristol? Any other high-end supermarkets we should consider?

Sgee: Harvey Guss is duly noted; I've heard others recommend that place as well. We were actually considering just a simple cast iron cook, but were concerned about consistency. Wouldn't charcoal be even more fickle?

Monday, June 03, 2013 5:38:00 PM  
Anonymous Anonymous said...

Which Ralphs did you get your steak from? Where I am, in Westwood, the Ralphs nearest me (Westwood Village, Brentwood on Wilshire at Bundy) carry only choice beef for steaks. The Ralphs on Olympic at Barrington used to carry some choice and some select -- you had to check the label. Are you positive it was select at over $10/lb.? Note -- I sometimes see prime offered at Ralphs. And when a steak is the highlighted star of the weekly ad, the cost can get under $5/lb. for bone-in NY strip or rib steak.


Monday, June 03, 2013 9:01:00 PM  
Blogger underbelly said...

Nice test. I think your intuition about cooking to a higher temp is spot-on. I used to assume rare/bloody steak was better, but sous-vide has disavowed me of this. I've also seen blind tests which show that everyone, regardless of stated preference, chose medium-rare.

Another thing to consider is that with relatively short cooking times for thick steaks, the interior will not reach the temperature of the water bath. That final 1°C can take hours. This is why chefs, including the Modernist Cuisine team, typically set the bath 1°C above the desired temperature.

For high quality steaks, I usually set the circulator to 55°C/ 131F. This brings the center of the meat to 54 / 129 in a reasonable time. It's pretty spot on for melting the marbling, bringing out the beef flavors (which you do not get at 124), and getting the texture right.

I'm betting you'd like this more for all the steaks, and that it might possibly change the rankings a bit.

Thursday, June 12, 2014 9:01:00 PM  
Blogger #gingerdaddy said...

I want to read your site but I can't. Every time I touch the screen it loads a new page. I was just about to find the results of your steak test before I lost my mind. I'm using Chrome, will be switching to valium.

Saturday, September 17, 2016 2:08:00 AM  

Post a Comment

Subscribe to Post Comments [Atom]

<< Home