SBI! vs. Wealthy Affiliate Review: WA Proof of Success… or Failure? Part 2
In Part 1 of this 3-part series, we recounted our discovery of fake SBI! reviews by Wealthy Affiliate and our subsequent deep-dive into an elaborate, high-volume program to generate fake reviews by Wealthy Affiliate (WA). These fake reviews all proclaim WA as superior to SBI!.
The keys to this misleading program are…
- the circular nature of “buy make-money to sell make-money” (over ⅓ of Wealthy Affiliate clients end up in “make money”-related niches). This turns customers into affiliates who replicate the process. Good for the company, bad for its client.
- the repetition, without proof, that WA is the best. This claim…
- it convinces affiliates that they do their visitors a service by recommending WA
- it convinces “freemium” customers to convert to paid
- it continues to convince customers that WA is the best.
But was this unproven claim true? We constructed a rigorous study to find out. Part 1 closed with a small peek into The Study’s results. We saw that…
- SBI! dominates at every level of success — the higher the level of success, the more SBI! dominates.
- Wealthy Affiliate only dominates in failure.
The conclusion is obvious: Thousands, perhaps tens of thousands, of everyday solopreneurs have been misled by fake SBI! reviews to a product that has a much lower chance of success and higher chance of failure.
This is wrong. We (SBIers and SiteSellers together) cannot let this stand.
In this second part, we’ll cover:
- Preamble: The Purpose of Part 2… “Claims” vs. “Results”
- Why Debate When You Can Debunk?
- Stand Up and Be Counted
- How Do You Bust an Illusion?
- The Study
Preamble: The Purpose of Part 2… “Claims” vs. “Results”
There are a lot of scams in Internet marketing/”make money.” Unless you know us well, you have no reason to believe the results of The Study in Part 3. And if you don’t believe them, then they aren’t results — they’re just more outlandish, unsupported claims, the kind of stuff you see everyday — to be ignored.
That’s why Part 2 is critical, even though it’s the “boring member” of this trio of posts. So, the way I see it, you can approach Part 2 in one of two ways…
- Skip this entire part and believe that The Study is rock-solid. Do this either because you do know us, or because you figure that others will pick it to pieces if it’s weak (check the comments below).
- Read it all, taking your time. Having been a physician and taught medicine, I’m trained in building and dissecting studies. I see no weaknesses.
We designed it to deliver fair, objective results. But you need to assure yourself of that. Either way you proceed (1 or 2)…
It’s important to understand and trust this as a well-designed study…
- “Claims” are unproven puffery (e.g., Wealthy Affiliate’s claim that there’s nothing else out there as good as it is).
- “Results” you can take to the bank. When you trust The Study, you can trust its results as fact.
Part 2 is written for folks who love to tear into material like this. I’m looking forward to intelligent comments and even some interesting debate. Like a good medical study, its construction must survive scrutiny…
That’s when results become facts that you can trust and act upon.
Why Debate When You Can Debunk?
Hundreds of fake SBI! “reviews” bait-and-switch prospective SBI! clients to Wealthy Affiliate as the #1 recommendation over SBI!. This made no sense…
- Our review of Wealthy Affiliate’s product and program convinced us that our success rate had to be higher.
- And their total lack of proof was suspicious (we figured that they would if they could). Even Wix has some success stories with domain names included (their traffic rankings are not great, but at least Wix tries).
Wealthy Affiliate reviews are, we feel, damaging our reputation and sales through misleading techniques that lead prospective customers and affiliates to believe claims that we think cannot be true.
We could go through every false statement of all those fake reviews and debate them in detail, showing how they range from flat-out lies to honest mistakes (and everything in between). That’s a lot of work, and many would get lost in the details.
Why bother? If we’re truly better, there’s an easier, more effective way…
Debunk their claims in a single blow. It all falls apart if we can objectively prove what was already obvious to us.
This would require a head-to-head study.
Only a rigorous, objective study could factually deny the claims of Wealthy Affiliate and the countless reviews of its huge affiliate base.
Until we discovered the factual truth…
- Wealthy Affiliate may know (from its own numbers) how badly its clients actually do, but no one else is privy to that data.
- WA may even realize that they were giving affiliates justification to create misleading reviews of SBI!. Their affiliates could not know that, either.
- And prospective customers (“pre-SBIers”)? Most would be fooled. They search for information because they don’t know, after all.
Even a cynical reader could be convinced after finding several reviews, all with the same conclusion, each with its own (faulty) information. Few would realize they had fallen into a WA sales funnel, not onto true reviews.
The bait-and-switch, fake review process is particularly damaging to SBI! and future solopreneurs. Why?
- Of the 175 companies that Wealthy Affiliate provides its affiliates to write reviews about, we are one of the most, if not the most, direct competitors. Even their process seems to mirror our own, albeit a “lite” and somewhat outdated version.
- Starting SBI! is a serious decision, so people search for reviews during their due diligence. If we don’t stop this, misleading reviews by Wealthy Affiliate will ultimately strangle SBI! as the number of fake reviews grows.
Part 1 summarized the entire scheme. It also gave you a peek into The Study, which exposes the reality of Wealthy Affiliate’s near-zero success rate (SBI! is 3300% better) and high failure rate (87% of Wealthy Affiliate sites are in the worst failure category of all).
The Study, as you can see, is essential to establishing the truth. After all…
- Numbers ignore what Wealthy Affiliate claims.
- And they ignore what we believe, too.
- Numbers simply… are.
However, numbers speak their own language.
They need interpretation. And they surely need help speaking loudly and clearly. We, and thousands of SBIers, must get the word out. How?
- Real users must tell their real stories on their sites. We have a tremendous amount of proof on our site. We know it’s all true, but it seems to be more believable when told off of the “vendor’s” site.
- We must get the truth to prospective SBIers and existing WA users/affiliates.
- We’ll need the help of honorable marketers who understand SBI!’s value to support our unique contribution to solopreneurship.
So that’s our plan. What Wealthy Affiliate has put online, using our own name against us with fake SBI! reviews that contain false claims and a worse recommendation (based on Study results), is wrong. It’s time to…
Stand Up and Be Counted
SBI! is the last bastion of everyday solopreneurship, the only all-in-one product where everyday people build extraordinary businesses and win… every day.
We have all worked too hard and too long, building an amazing, proven track record, to let this fake review attack by WA stand. We call on the moral compass of…
- Wealthy Affiliate’s leadership — Do what’s right. Make a better product rather than tear the better one down. Stop misleading affiliates into making false claims.
- WA’s affiliates — Most of you are failing in the “make money” space. You have a much better chance in a niche you know and love. SBI! helps folks find those niches, whether they reach back 40 years or look in the back yard.
And to all successful WA affiliates — you won’t earn as much with SBI! (we don’t pay as much to keep the price of SBI! accessible). Don’t miss Part 3…
You’ll understand the full weight of leading good people down a bad path. But you can’t put a price tag on doing what’s right and feeling proud of that.
Please contact us at SiteSell Support and ask to speak to me personally if you have good search rankings and an upset conscience.
- To the 87% of Wealthy Affiliate’s customers who are failing (per The Study) — Think back to how you found WA. Almost the entire affiliate program is about writing fake reviews. I promise you only two things…
- SBI! raises your likelihood of significant success by 33x (more in Part 3). That doesn’t mean you’ll be 33x more successful (although SBI! does improve level of success, too). It means that a solopreneur who is starting out with SBI! is 33x more likely to build high traffic.
- We’ll never mislead you, waste your time with “busy work,” etc. The total focus is putting every second into moving your business forward. You may be “used to” Wealthy Affiliate now, but that won’t move you any closer to the goals that matter to you. SBI! is more likely to do that, partly because we do not use people. We enable them.
- To all who believe in the solopreneur’s ability to reshape life by building an online business, please support the only product that gives the solopreneur the best chance to achieve a life-changing goal — a dream.
Please read on, and don’t miss Part 3. After reading The Study’s full results, please support us to assure the future of everyday solopreneurs. The numbers are so staggering, there’s no need to use SBI! to “get” its impact.
If you reach solopreneurs, send them to this series of blog posts, particularly Part 3. Do your audience, those just starting or who have yet to succeed, the best favor you possibly could. Do it just because it’s right, or join the affiliate program (and still do it because it’s right!).
We’ll be marketing this message aggressively, too, of course. We have no choice. If left unchecked, Wealthy Affiliate and its fake reviews will slowly, inevitably choke the best thing solopreneurs have going for them.
They do it merely by placing fake reviews at the most critical part of the process, the point where people make their decisions. That’s devastating, all the more wrong when you see how dismal their results are. If left unchecked, that kills…
Not today or tomorrow. We have large cash reserves for rainy days or potential acquisitions. But frankly, it’s a tough, non-productive way to spend our time. We’d rather that WA see the light so we can all move on. One way or the other, though, we won’t be leaving this immoral strategy unchecked…
“The Truth Will Out!”
The only absolute truth is a head-to-head-study, one constructed so solidly that it will withstand the utmost scrutiny. That brings us to the main purpose of this post.
Part 2 (this post) focuses on an explanation of The Study — its goals, construction, methodology and so forth. We owe it to the serious reader to provide all the information needed to evaluate The Study and judge whether its results are a fair reflection of reality.
A well-built study must be…
- objective (conclusions must be driven by data from agnostic third parties beyond our control)
- rigorous (constructed with care, it must minimize any issues that could give one product an advantage, e.g., selection bias)
- statistically significant (results cannot merely be a statistical “fluke”)
- reproducible (anyone can do it and end up with the same results*).
A study repeated a few weeks later might get 550 and 60, respectively (a ratio of 9.2:1), or 450 and 40 (a ratio of 11.3:1). In all three cases, the ratio would be enormously in favor of WA. All three of these results would have the same “huge difference” conclusion.
With that in mind, let’s get started. This study was born out of an unusual situation, which we’ll summarize quickly…
How Do You Bust an Illusion?
As Part 1 outlined, Wealthy Affiliate has concocted an elaborate “circular” system and training program that turns its affiliate program into a “fake review machine.” But most people, we believe, are inherently good.
So how do you get so many people to support (what we believed to be) a poor-performing product without proving success? The answer is a testament to the goodness of people (who tend to believe what’s written) and the power of false repetition…
In other words…
1 ) If you say it often enough to your affiliates, they believe that they provide a valuable function by recommending the product.
2 ) And if thousands of affiliates repeat “SBI! is bad, Wealthy Affiliate is our #1 pick” often enough in their fake reviews, well…
Prospective SBI! clients come to believe it.
Affiliates can justify unbalanced, even inaccurate content, if they believe that they do it all as a good service to the visitor to that web page.
Wealthy Affiliate accomplishes all of this despite a real track record that’s dismal. We didn’t realize how dismal it is (our guess was that SBI! would outperform WA in success by 400-500%, based on our review of the product — flawed, out-of-date, etc.).
This was our working theory. Now that we know the real numbers, the only question we can’t answer is how this was kept secret for so long.
One thing, though, that’s not a secret, is this…
Fake Reviews Work
We shall be following this series of three posts reviewing Wealthy Affiliate with a series on fake reviews (hey, we got interested!). Putting WA aside for a moment, fake reviews plague any niche where there’s money to be made.
The old school topics of Internet marketing (e.g., affiliate marketing, MMO, etc.) are particularly infested. After 20 years of “make money” schemes and affiliate marketing (which was mostly honorable 20 years ago), the “dark side” (which I wrote about long ago) was inevitable.
Many prospective clients have become too cynical to recognize the real thing. Yet they, along with less suspecting people who are interested in SBI!, can be taken in by elaborate networks of fake reviews all recommending Wealthy Affiliate.
With the stage set, let’s get into The Study.
The Study (a.k.a. “The Solopreneur Success Challenge”)
The goal of this study is to determine which product is more likely to enable solopreneurs to build profitable online businesses. We’re only looking for big picture, absolutely conclusive/meaningful differences. For example…
- Does one product make a solopreneur significantly more likely to succeed than the other? A result that shows WA makes a solopreneur 250% more likely to succeed would be meaningful, while 50% more likely to succeed would be suggestive. A 5% difference? Meaningless.
Important: If you’re unfamiliar with the use of Alexa, SimilarWeb and SEMrush to determine site traffic, please click here for the definitive review of all three.
- We’re also looking for big differences in the level of success. There’s little difference between a site with a traffic ranking of 900,000 or 1,000,000 at SimilarWeb, for example.
However, both of those sites are much better than sites with a ranking of 10,000,000. Similarly, a site inside of 100,000 (extremely rare for solopreneurs) is a significantly bigger success than one at 1,000,000 (which itself is excellent).
For this reason… The definition of success is based on ballpark traffic estimates (see “Definition of “Success” below). Any site that ranks in the Top 1,000,000 at Alexa or SimilarWeb (SW) is in an excellent ballpark.
So, if Wealthy Affiliate places 3 times more solopreneurs into a high-traffic ballpark (e.g., <1,000,000 at Alexa — aka “Top 1M”) than SBI!, that is significant. A 10% difference is virtually meaningless.
- The definition of failure, too, is based on ballpark traffic estimates. A site with a traffic ranking of >30,000,000 at Alexa or SW has zero or near-zero total traffic. It’s in a significantly inferior ballpark than, say, a ranking of <1,000,000.
- Finally, what’s the pattern of the difference from most successful to least? If one product has a definite edge over the other, a clear pattern should emerge from the best performance level to the worst.
In summary, if SBI! and Wealthy Affiliate place similar percentages into similar ballparks, this study will be inconclusive. We are especially interested in the results for those sites ranking at the extremes…
- inside the Top 1,000,000 sites (out of a total of 170,000,000 active sites, according to Netcraft)
- beyond 30,000,000 (also referred to as “>30M” — we call it the “Invisible” ballpark in our review).
Wealthy Affiliate has claimed, through thousands of affiliates, of which a small number generate a high percent of SBI!-related search results, that it’s the best, meaning that it delivers the best chance for solopreneur success.
If Wealthy Affiliate really has found a new and innovative way of delivering solopreneur success online, we will have two options…
- admire the innovation and help SBI! members move their sites to WA
- admire the innovation, learn from them, and apply similar concepts to SBI!.
As SiteSell’s founder, I have assured everyone (SBIers, SiteSellers and you, the reader) that should a significantly superior system ever emerge at a similar price, we would help SBI! customers migrate over to the new company if…
- the majority of SBIers agreed to move
- this innovative new company helped automate the migration of SBIers to their platform
- the innovation was patented, or had technical features that made it too difficult for us to incorporate quickly
- management had the same “solopreneur first” attitude, a philosophy that “we do well when solopreneurs do well.”
The reason is simple…
We are dedicated to solopreneur success.
We believe in the tortoise with BAM, the one who dares to stick that vulnerable neck out.
The Internet provides the opportunity to build a successful business to the “everyday solopreneur” who is ready to work…
That venture grows profits and equity that enable him/her to reach goals that are “life-level” important (e.g., anything from contributing to a special family vacation to quitting the soul-sucking day job).
People are capable of amazing things. It is, however, too difficult/noisy for even the most BAM-laden person to figure out “the Internet.” Too many major stumbling blocks stop most people. SBI! was born on one concept…
Remove the stumbling blocks. If you clear the pathway, you can enable people to focus on the only thing that matters — building their niche-focused business.
That may sound idealistic, but the theory works! SBI!’s constantly evolving process of C T P M works better than ever. Even after 15 years, SBIers amaze us..
The current stories that we publish in our blog (complete with useful takeaway lessons) thrill us all at our meetings. Working on SBI! is a great job!
We make an important difference in people’s lives. That is a mission!
Don’t get me wrong. All credit goes to the solopreneur. S/he is the one who took the still-unconventional chance, the one who sacrificed TV time in the short run to achieve an important goal for the long one. Given all that…
We’d have no interest in being second best. We would be doing solopreneurs a disservice if using SBI! decreased their likelihood and level of success. If an equivalent service came along…
- We’d love to work for it.
- SBIers would almost surely be happy to move over.
“What about Ken?” you ask. Ha ha! Janice and I would be right back to doing what we were doing when we started SiteSell with two other people who still work here….
She and I would hop on the first jet to somewhere far away.
That will happen at some point, I’m sure. Actually…
We challenge other hosts with 10,000 or more solopreneur websites to replicate this study with their own population of solopreneurs. Please publish your winning results and contact us.
As strongly as we feel about the “solopreneur who dares,” no company like Wealthy Affiliate can be allowed to strangle the everyday solopreneur’s current best chance at living “The Individual Dream” (an update of the lost American Dream, except it’s digital and open to anyone, anywhere).
We have too many big plans for SBI! to let that happen. We all profoundly believe that solopreneurs with BAM can succeed online. The SBI! Advantage has grown and continues to grow.
Our bright history was largely search-based. Its brighter future will continue to have search as an important component (thanks to Google’s RankBrain). Social media (where the edge goes to the passionate individual — if well done) and the increased total activity due to mobile (both social and search) play growing roles.
Many trends have emerged since SBI! launched in 2002. We’ve been there every step of the way, evolving the process to meet the mission with growing results…
- Enable solopreneurs to lead lives of freedom or to accomplish smaller goals (e.g., a stay-at-home spouse contributing to a college fund). The best, least risky way to do that is through an online business.
- Solopreneurs make up the amazingly diverse ecosystem that we call the Internet. The SBI! process helps solopreneurs add to it by converting unique niche-based information and perspective into a profitable online presence.
Achieving that success is dependent on two factors…
- BAM: Solopreneurs with high-win probability have BAM.
- Focus: Solopreneurs are time-limited. A significant cause of failure is “I just don’t have enough time.” However, if you maximize their time-efficiency, they are capable of extraordinary results. To do that means to…
- Remove the high learning curve of “figuring it out.”
- Provide a proven step-by-step process.
- Include tools that handle the parts that would be difficult, tedious or otherwise time-consuming.
- Deliver guidance and support as necessary, including through a “help and be helped” community.
- Constantly keep up on “the latest,” evolving the process and software, allowing the solopreneur to focus on keeping up with his/her niche, not on the daily firehose of information in tens of fields related to online business-building.
No pretend, self-serving wannabe can be allowed to extinguish that flame.
And that brings us back to this study. As said earlier, a well-built study must be objective, rigorous, statistically significant and reproducible.
The first three factors are important, of course — they establish the results as meaningful. But we feel that it’s especially important for it to be reproducible. That’s why we’re publishing the exact methodology — if you don’t believe the results, you can do it, too.
Let’s get started…
Definition of “Success”
We must define success before we can measure it.
Solopreneurs start businesses to fulfill important life goals. Those businesses must be profitable to delivers such goals as…
- quit the day job and replace that income, “putting me in charge of my life”
- supplement my retirement to ensure the same or better standard of living
- save enough money to fund college, or perhaps an exotic vacation
Big or small, goals with emotional impact motivate. Generating profits from a business is merely how to achieve those goals.
So how do we define “success?” Attainment of a long-term goal? Income?
To truly document success, we need hard data that can be verified by anyone. That eliminates…
- Attainment of goals. The definition is soft, and anyone can be making good progress toward a goal without having reached it. Impossible to verify.
- Income. Most people don’t like to quote exact income figures, and those who do might exaggerate. Getting permission for audited results is not practical. And then there’s the issue of enabling others to verify the results.
The best way to document success is by determining organically grown (i.e., free) site traffic. Anyone can buy traffic, but there’s a steep learning curve and it requires the solopreneur to have a product with a high enough margin to justify the expense.
Growing free traffic accommodates all types of monetization, from the lower-paying passive income models (e.g., AdSense and affiliate programs) and higher-paying, more active models (such as creating and selling your own product).
The same amount of traffic can generate different amounts of income, but that’s a choice that reflects the goal. Whatever someone’s monetization mix may be, it’s fair to say that income is proportional to traffic. And, as we’ll see, traffic can be easily verified.
Traffic is our best proxy for success. The Study will, therefore, measure free, organically grown traffic as its indication of success.
How Does The Study Measure Traffic?
So how do we measure the traffic of websites for this study? Possible services to consider include…
1 ) Google Analytics (GA): Securing traffic graphs from thousands of owners is impractical. In any event, screenshots are easily photoshopped. They’re impossible for observers to verify, and there’s no way for you to reproduce the study.
2) SBI! and WA traffic analysis: We know how much traffic each SBI! site receives because they’re hosted with us and we have their log files. Traffic analytics software delivers accurate traffic measurement. The situation is the same for Wealthy Affiliate. However, a problem arises…
That type of data is the confidential information of the site owner. Since each party would want to see the individual traffic of each site, that’s a problem. It would also need the cooperation of both parties — an obvious issue since one party would likely refuse. And in any event, you would not be able to verify or reproduce The Study.
Luckily, there are several providers that can give good insight as to how much traffic any given site receives…
Alexa, SimilarWeb (SW) and SEMrush all use different techniques to sample part of the web population. They extrapolate their findings to represent the behavior of the overall web population.
The choice of these metrics providers is discussed in “Discussion of Our Choice of Traffic Metrics Services” below, as part of Methodology…
In order to compare apples to apples, The Study must use the identical methodology to analyze SBI! and WA traffic performance of active (“real”) sites hosted at Solo Build It! and Wealthy Affiliate.
We performed The Study on April 3, 2017. We repeated it in early May with near-identical results. Here’s the step-by-step process to perform it yourself…
- Get the list of websites for WA and SBI! from whois.domaintools.com (or any source that provides all sites hosted under each name server.
32,582 domain names were listed under “MYWAHOSTING.COM” (WA’s nameservers are NS1.MYWAHOSTING.COM and NS2.MYWAHOSTING.COM).
16,973 domain names were found under “SITESELL.COM” (SBI!’s nameservers are NS1.SITESELL.COM and NS2.SITESELL.COM).
- Reduce the sites to a list of “real” sites. For example, the following types of sites had to be eliminated….
- redirects to other sites
- single page sites (under construction, not configured yet, etc.)
- unreachable/does not resolve
We define a “real” site as one that has a home page and at least one link to a second page on the same domain name.
16,814 “real” sites for WA and 9,948 “real” sites for SBI!. That means…
- 15,768 Wealthy Affiliate domain names were not real sites at WA.
- 7,025 SBI! domain names were not real sites at SBI!.
- Get traffic rankings for each site. We obtained, using the APIs of each company, the Alexa ranking, the SimilarWeb (SW) ranking and the organic search traffic estimate from SEMrush for all “real” sites.
- Finally, we organized the data for presentation (see “Study Results” below).
Discussion of Our Choice of Traffic Metrics Services
When measuring the traffic of sites you don’t control, you do not have access to those sites’ Google Analytics or log file analyzers. So you must use a system that…
- samples a large subset of the population, and then
- extrapolates the data to reflect reality.
We chose Alexa, SW and SEMrush for this study because…
- they are the three largest, free metrics providers that do this
- each of them uses a different technique to sample the web population.
There are paid tools that are supposed to provide better data (e.g., Hitwise, Quantcast, comScore). However…
- any difference would not change the “ballpark accuracy” that we seek
- the expense would make this study less accessible for those who want to reproduce our methodology.
In this comprehensive review of Alexa (with coverage of SW and SEMrush), we found that Alexa provides a reasonable ballpark estimate of a site’s traffic.
We’re aware of a widespread misconception that there’s a large sampling bias for sites whose visitors are more likely to use the Alexa toolbar in the browser. If anything, that bias would negatively affect SBI! site traffic measurement. We accept that condition. Please read our Alexa review to see why this perception is outdated.
All systems using indirect methods will have two limitations…
- Scatter. This is the deviation of actual traffic from the traffic numbers derived by a sampling technique. It increases if the technique has a small sample set of users that it tracks. All three sources use huge sample sets.
The more traffic that a site receives, the lower the scatter (relative to the traffic). The less traffic that a site has, the higher the scatter.
That said, our review shows that these tools are valuable ways to ascertain ballpark estimates of even a single site with good reliability. Since the study measures 10,000 SBI! and 17,000 Wealthy Affiliate sites, scatter tends to average out to close to 0.
- Sampling bias. The act of sampling can introduce a traffic over-estimation for sites that receive visitors who are more likely to be exposed to the sampling source.
For example, if SW tracked data from an ISP in Canada as its main source, Canadian news sites would suddenly appear to “become” the most visited sites in the world.
Sidebar: SW does use ISP data (and other types of traffic sources), but it chooses many ISPs that provide a representative global sampling, adjusting its algorithm to account for known bias.
Here‘s how the study minimizes these issues…
When measuring hundreds of sites, and especially well into the thousands, the impact of scatter averages out to near-zero. That’s because some sites scatter higher than reality, others lower. But when taken together in large numbers, they average to close to zero.
Similarly, the number of sites in each traffic range is reliable due to the number of sites in each grouping of traffic ranges.
For example, a traffic ranking of 1,000,000 at Alexa could be as high as 1,200,000 or as low as 800,000. If there’s only 1 site in the “Top 1,000,000” grouping, it’s true that there may actually be 0 or 3 that “deserve” that placement (if we had access to the actual traffic from GA).
But if there are 50 sites in that group for SBI! and 100 for Wealthy Affiliate (ignore the fact that WA has 70% more sites for the moment), that would be a 100% increase, a significantly large and reliable conclusion…
Why? Some of the sites at 900,000 might “deserve” to miss the “Top 1,000,000” maximum.
But some of the others at 1,100,000 would “deserve” to be in that group.
In other words, when you have an appreciable sample size…
“It all evens out in the end.”
On the other hand, scatter is greatest when traffic is lowest. If a site’s Alexa ranking is 20,000,000, it could jump to 10,000,000 with just a few more visitors. Neither one has particularly great traffic, though. And again, when you have enough sites in each group, the “total effect” of scatter is minimized.
Finally, when a site is greater than 30,000,000, you can be sure that site is virtually invisible online. It means that a vast number of toolbars, browser extensions, ISPs, etc. did not detect a single visitor. Even if they missed a few, that site is basically “Invisible.”
So when Traffic Rankings are >30,000,000, the total number of sites that “belong” there becomes increasingly reliable, especially if the number of sites is high.
That dash (see right) means “no data,” which means that none of its huge user base has encountered that site. The odds of having even a few visitors per day is negligible.
If SBI! had 400 sites in this final “Invisible” grouping, compared to 200 for Wealthy Affiliate, it could be stated with statistical certainty that you are less likely to fail with WA.
Summary/Conclusion: The sample size (10,000 “real” sites for SBI! and 17,000 “real” sites for WA) provides good traffic estimates by averaging out scatter. The extremes (under 1,000,000 and >30,000,000) are particularly reliable.
2) Sampling Bias
This bias is “built into” the sampling method. So high sample sizes won’t “average out” bias.
The most common example of sampling bias is connected to Alexa. Prior to 2008, Alexa used a single toolbar to track its users (its own Alexa Toolbar). That toolbar was more likely to be used by the tech-savvy (e.g., Internet marketers). Result…
If a site received those types of visitors, its traffic reporting would deliver estimates that were higher than reality. This was obvious and well-documented at the time.
However, in 2008, Alexa expanded its sources to 25,000 toolbars/browser extensions/etc., growing and diversifying its user base. It has also made algorithmic releases to reduce bias. In discussions with Alexa, we learned that its toolbar now accounts for less than 4% of all traffic measured.
Despite these improvements, negative reviews of Alexa persist. The continuing widespread prevalence of blog posts that conclude Alexa bias remains appears to be a failure to do original research. Facts that led us to that conclusion include…
- no mention of the 2008 addition of 25,000 sources and diversification of users
- the recurring (out-of-date) mentions of the “single toolbar” as being the problem
- the lack of original research/new data
- failure to explore its valuable uses, regardless of bias.
No motive is imputed. While reviewing post-2008 articles, we found several hallmarks of a predisposition to find the negative and ignore the positive. One compelling example….
Some studies, published well after 2008, quote Alexa’s disclaimer that only traffic rankings of of 100,000 or better could be considered reliable. Such a single site limit requires remarkably high traffic.
Those reviews fail to think about this as unreasonably high. Open minds would look for possible reasons for this extremely high-traffic “reliability limitation.” We traced the first occurrence of this disclaimer back to 2003. In other words, Alexa neglected to update it in 2008!
An earlier occurrence may be on a different URL. But since we could not find anything earlier, we’ll use 2003 to work out a correction factor. We must consider the following in 2003…
There was 1/10th the number of sites (just 18,000,000), and there is now 145X more traffic. That works out roughly to our own empirical experience that 1,000,000 is a reliable indicator for ballpark estimates of a single site (see our review for details).
Finally, in this study, all categories are significant due to the volume of sites in each.
The literature lacked a rigorous post-2008 Alexa study that demonstrates how much, if any, toolbar bias remains. New articles basically reinforce what came before, each new one building on the previous literature, right up to this year.
This is fully discussed in the review.
Our sole purpose for spending some time on this topic is to make the point that indirect traffic measurement has come a long way since the turn of the millennium.
To summarize, conclusions of bias in present-day articles about Alexa seem to reflect the recycling of information from previous reviews and articles. The literature needs a rigorous study with original research to assess the effects of Alexa’s 2008 and subsequent improvements to its traffic base and algorithmic corrections for bias.
We believe the information contained in our comprehensive review to be the most accurate and reliable information about Alexa, including how and when to use it with SW and SEMrush.
Conclusion: Per our Alexa review, Alexa is a valuable tool for ballpark estimates of a site’s traffic. If any bias remains, it favors WA.
Read the entire analysis and learn the useful techniques in the review of Alexa (including how and when to use it with SW and SEMrush to increase the certainty of ballpark traffic).
All three tools are particularly valuable at the extremes (highest and lowest traffic ballparks). The data shows that any one of the tools is useful to determine ballpark estimates (7 ballparks — see the review) of site traffic, as long as you understand how and when to use it.
Now that we’ve covered sampling bias, how do we minimize it? We noted earlier that measuring thousands of sites won’t reduce the bias.
So what minimizes it? The use of all three tools, each of which depends on a different set of sample sources, will help to cancel out each other’s bias. For example, if Alexa does still have some “tech-savvy” bias, but if SW’s bias is a geographic one (e.g., overestimating India-related sites), considering both dilutes each other’s bias.
Finally, we add a third tool to further “dilute” any bias…
Adding SEMrush, which focuses on search traffic for the purposes of our study, adds further confirmation if its numbers correlate with the Alexa and SW rankings. High search traffic would confirm high Alexa/SW total traffic. A discrepancy (low SEMrush, high Alexa/SW) would suggest other sources of traffic, which could be verified by using the “Other Metrics” section of our Alexa review.
Sampling bias is greatly reduced by using all three tools. Each of these traffic-measuring tools uses different sampling methods. This diversification reinforces confidence in the ballpark estimates for a single site. For hundreds of sites, results are indisputable if they correlate with each other.
The results are presented as bar charts for each of the three tools. If the three sets of results are similar to each other, you can feel considerably more comfortable than relying on any single traffic-measuring tool.
For example, if Alexa shows that Wealthy Affiliate reduces failure (>30,000,000) by 30% and increases success by 50%, that’s likely conclusive, given the number of sites.
However, if you believe that the original Alexa bias is still present, you could argue that WA benefits because ⅓ of its sites are related to “make money,” some form of Internet marketing, reviews of online business-related products, etc. That would be a huge boost for their traffic. What to do?
Check results using the other two tools. If they confirm Alexa’s findings, the reliability is certain, especially given the number of sites involved, which reduces scatter.
And that’s it! You now fully understand how we performed The Study. If you have criticisms of the decisions we’ve made or how it’s constructed, please leave them in the comments box. We’ll reply.
We’re ready to investigate the results!
Does the Study Have Weaknesses?
Are There Rebuttals?
As we’ll see in Part 3, this study not only exposes Wealthy Affiliate’s long-kept secret of near-zero success and high failure. The Study’s process can be used to compare any group of solopreneurs using sitebuilders such as Wix or that are hosted by any company such as GoDaddy.
It will be an advance in assessing solopreneur performance in general. Providing solopreneurs with objective data like this will help them select the product that gives them the best chance of success. And that, in turn, will force businesses that want that business to become more success-focused. Remember…
Companies like Wix and GoDaddy are hard-core big businesses. Don’t get us wrong — we have nothing against capitalism. These companies make entertaining Super Bowl commercials. But the solopreneur needs to have higher standards.
No company is ever likely to create another SBI!, not without a push. It’s not sexy enough for VC-funded entrepreneurs, and it’s just too much work to cater to a small percent of the population, those who are ready to work to change their lives.
No one wants to say that, of course. So, as long as a Jason Statham car chase convinces solopreneurs to sign up, why create another SBI!?
We hope the publication of data about other companies helps to drive that home.
For now, as the defender of the solopreneur, it falls to us to reveal what we have long known and said… the failure rate of solopreneurs who try to build an online business is dismal. Yes, we know…
As Wix notes in every Super Bowl commercial, it’s easy to put up a site. But then what? They don’t mention that “turn it into a business” is the hard part. This process will be invaluable for documenting the true state of solopreneurship. But I digress…
Let’s wrap up this discussion…
We don’t expect anyone to find a weakness of significance in The Study. It’s solid.
We do expect WA (and others with vested interests that cost too much to let go) to reply with weak excuses. We foresee, for example, the following…
“Not all WA customers are hosted by us.”
That however, is true for SBI!, too. But the vast majority are hosted by both of us since it makes little sense to pay for the product and not use the full, integrated version. The sample set is not only representative of reality, there’s no reason to expect “off-host” customers to do better or worse.
It’s possible to be a “customer” of either product without using its sitebuilder, but it’s sub-optimal. For the purposes of this discussion and The Study, we define “paid customer” as someone who uses the product fully, including the builder.
Regardless, we’ll be glad to debate and air out this topic fully. Solopreneurs deserve the truth. Meanwhile, let’s head into Part 3, where we’ll learn an important lesson…
Repetition of an untruth seems to be quite effective. But generating countless fake reviews does not impress a well-constructed, rigorous study that measures success and failure rates.
We are glad to finally have a single reply to the variety of “urban myths” that have accumulated over 15 years. We even have part of our site devoted to the myths.
Competitors, and those whom we have “called out” (e.g., that large group of folks who believed in mini-sites), and even former “impossible clients,” have been prolific!
Regarding “Impossible Clients”
We take valid comments from SBIers seriously. SBIers have driven improvements and even new features.
But every company has impossibly difficult clients. We have always fired those who repeatedly abuse our Support staff or who troll our forums.
We accept the “cost” — some get angry enough to create the most amazingly nasty sites. So be it. They always tell the story their way — 2% truth, 48% fiction and 50% coloring outside the lines. They never mention that they were forum trolls.
Rather than get sucked into debating their “bait,” we can now simply “de-bait.” The results of this study say it all when we reply…
“How can you say ‘SBI! _____FILL IN MYTH HERE_____’ with these results? How much can it matter, even if some small part of that is even slightly true?”