First, I will provide some background details of what prompted me to write this post. Next I will describe why I believe ASR (short for Audio Science Review) got golden samples. Finally, I will conclude with my overall take about Audio Science Review and provide my general advice to people new to ASR.
Background
I joined as an ASR member on Sep 15, 2020. User name is pdxwayne. The main selling point of ASR is that as long as Amir (owner of the site) measures a device as transparent and recommended it, it will sound the same as all other similar devices deemed transparent by him. This is a very enticing selling point. It means a $100 DAC (Digital Analog Converter) would sound no different than a $2000 DAC, as long as both are deemed transparent by him. I can enjoy the best quality sound and save lots of money by buying the lowest cost transparent devices.
During the +1.5 years as a member, I purchased 9 DAC and Headphone Amps (some returned due to issues) based on Amir’s recommendation. Unfortunately, majority of items I bought either not working properly, or sounded different than expected. Note: Readers can search for “bait and switch issues” in ASR to see the descriptions of issues I encountered.
In one of my ASR posts, where I discussed Topping D30pro issues, one member shared a link from a Chinese reviewer. The review is at https://www.mobile01.com/topicdetail.php$f=348&t=6342681
The reviewer made this comment in his post:
使用Altera CPLD 5M80ZE64C5N作CLK&數位訊號重整，控制的MCU使用ST STM32F030R6T6
運氣不好拿到兩顆都rework的板子，我是能接受rework但好歹也清一下flux吧= =”
Basically, the author said that “unfortunately, both chips are reworked chips.” This created a red flag to me.
Combining my own bad experiences over 1.5 years and the comment from Chinese reviewer, I started to wonder if the devices massed produced for retail sales (at least those I bought from SMSL, Topping, and Gustard) could contain parts that are not as good as the original reviewed samples. I started a thread in ASR called “bait and switch issue?”
Clearly, starting a thread with such title is not something well tolerated by the people behind ASR.
Amir (the owner) first response in my thread is this:
You have a concern that I do not. So if you want something done here, you should do the gathering and fund such a study. I have so much to test that I don’t see dealing with your doubt being a good use of my time. I can be bought however. 🙂 I will charge $200 per device to do this secondary testing. Get me as many as you like with a check for that amount (each) and I will test them all.
In subsequent conversion with a member, I mentioned if I should pay Amir to compare my devices with his “golden samples”. One moderator got really offended that I mentioned “golden samples”, closed my thread, and responded this way:
So you acknowledge you have received direct reply from Amir on your questionable call out challenge. You don’t like the answer. I guess you expected Amir to drop everything he is doing to allay your unsupported suspicions about some of your gear not performing up to spec? Then you accuse Amir of only testing “Golden Samples”. Which on its own merits is historically unfounded. You never give up. Just keep digging a deeper hole. Now you have slowly but surely dishonesty transformed this thread into a Troll Amir Thread! I no longer believe you are here as a good faith contributing Member. We have let this Thread run its course and allowed you to have a voice only to regret giving you the benefit of the doubt.
The moderator twisted my words and said that I accused Amir of “only” testing “Golden Samples”. Of cause I did not mean that. It would be unreasonable to make such claim. Amir indeed tested non golden samples in the early years. Amir will also test samples bought by regular members. But, for moderator to claim “historical unfounded” is only half truth. I will explain later why I believe Amir got golden samples.
Obviously, ASR is a business. It is very clear that sharing my bad experiences is bad for business. Further fair discussions are impossible in ASR.
I thought about how to response to such accusations from ASR. Then, I realized I have a web site that I have not been utilized properly for over a decade. I decided to update the website to use WordPress and create this post as my first WordPress post.
Why I suspect ASR got golden samples
So, what is golden sample?
To me, any sample that manufacturer can hand select and check for quality is golden sample.
What is not golden sample?
To me, any sample that Amir (the owner) could buy from normal retail stores like Amazon is not golden sample.
Let’s review the 9 items that I bought based on Amir’s recommendation. Let’s see if there is possibility of Amir getting golden samples.
*SMSL M500 DAC and HP Amp combo (bought and returned due to defective screen):
Amir posted review on Oct 28, 2019. Amir said “It was kindly sent to me by the company”.
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*SMSL HO200 Headphone Amp (bought and returned due to issue observed):
Amir posted review on Oct 24, 2021. Amir said “It was sent to me by the company”
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*Khadas Tone Board:
Amir posted review on Oct 10, 2018. Amir said “It is on kind loan from the company”
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*Topping E30 DAC:
Amir posted review on Mar 19, 2020. Amir said “It was kindly sent to me by the company for testing. Unfortunately I did not get pricing information”.
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*Topping L30 Headphone Amp (original version):
Amir posted review on Aug 10, 2020. Amir said “I don’t have official pricing on it”. Note: If he didn’t even know the price, obviously it is from the manufacture.
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*Topping L30 Headphone Amp (replacement version):
There was a recall of original L30 due to possibility of causing headphone damage.
Did Amir later on buy from a retail outlet and recheck the replacement version for quality: No
*Topping D30 PRO DAC (bought and returned due to issues observed):
Amir posted review on Feb10, 2021. Amir said “It was kindly sent to me by the company.”
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*Gustard X16 DAC:
Amir posted review on Nov 10, 2020. Amir said “It was kindly sent to me for testing by SHENZHENAUDIO.” Note: Gustard confirmed that the sample is preproduction sample and not production sample.
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*Gustard H16 Headphone Amp:
Amir posted review on Dec 6, 2020. Amir said “It was kindly sent to me by one of their dealers, shenzhenaudio” Note: See the note about Gustard X16. X16 was also sent by Shenzhenaudio and Gustard confirmed that the sample is preproduction sample and not production sample. Thus, H16 is also suspected…
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
*Sabaj A10H Headphone Amp:
Amir posted review on Aug 11, 2021. Amir said “It was sent to me by the company”
Likely to get a golden sample: Yes
Did Amir later on buy from a retail outlet and recheck: No
To summarize:
It is very clear that for all 9 items I bought due to Amir recommendations, Amir got his samples directly from the manufacturers or manufacturer representative. Amir never once did his due diligent to “verify” with regular retail samples. Readers can further check all other new devices from Topping, SMSL, Gustard, Sabaj, Loxjie, Khadas, etc., that Amir reviewed since 2019 until now. How many of them were sent directly from manufacturers or representatives? Since 2019, did Amir buy even one sample from retail stores like Amazon to further compare with his review samples?
Since 2019, where is the concrete data to support the moderator’s claim that “golden samples” is “historically unfounded”? Yes, Amir tested some devices he personally purchased back in 2018 and earlier. What about 2019 and on? For the rare one time during 2019, where Amir compared a member device with his review sample, titled “SMSL SP200: Do We Get Golden Samples For Review?”, there were differences in headphone channel imbalance. Yes, I repeat, there were differences found when he later checked with a production unit! Did he care/dare to do more such check with those DAC and HPA he received from manufacturers? Other than that one I just mentioned, none that I know of for these few years.
Based on all the examples I have shown so far, do you think there are enough “historical” data to support the moderator’s assertion that Amir received NO golden sample? Isn’t it ironic, a moderator of a so called “science” based web site was playing words game instead of using real data to support his assertions? I subsequently PM both Amir and Adam (the moderator) to ask why they are so sure they are not getting golden samples. Guess what, they could not give me even one example!
ASR is getting popular over the years. With great power comes great responsibility.
Did Amir do enough due diligent when recommending devices that he got directly from manufactures for free? When I shared my disappointments with my purchase experiences, did Amir show any concern? Answers are clearly “NO”.
Conclusion:
I hope I have shown reasonable explanations why I think ASR got golden samples.
My thought about people behind ASR? No longer positive. Their lack of concerns have turned a fan of the site into a critic of the site. Over +1.5 year as member, I followed Amir’s recommendations, spent thousands, and did not like what I get. I shared my bad experiences, asking questions, and ended up being called a liar and a troll by ASR moderator.
Is everything a loss? No.
I still appreciate these:
*I learned how to perform voltage matched blind tests.
*I was exposed to many blind tests, thus have a better sense about my hearing sensitivities. If still there, search for “Tracking my listening tests’ keyword in ASR to see all the tests I have done.
*I met a few very patient, knowledgeable members who are willing to teach. Members like “Blumlein 88” and “solderdude” deserve special mention! Thanks for all the helps over the 1.5 year!
*I learned about measuring with tools. For example, I performed many measurements using ADC and REW. One special tool I really like is DeltaWave Audio Null Comparator by member pkane. His tool deserves special mention. It is free from https://deltaw.org/. Paul, I really appreciate all your help and patient when I was figuring out the Topping d30pro issues!
*I actually like using Gustard X16 and H16 in my main stereo setup.
Finally, my advice to readers who are new to ASR: If you sense differences between devices, do your own due diligent when possible. Perform voltage matched blind test if possible. Do other checks. Do not let some “cult members” of ASR bully you into believing the differences you heard are all due to “imaginations/bias”.
Thanks for reading!