In total, i tested it with a scientific eeg device for 11 nights. As always, i do not want to waste your time, so timestamps are in the description below and also on the timeline Music Applause Music. For those of you that are new to the channel, my name is rob and i’m a postdoctoral scientist specializing in biological data analysis. In this video, i will look at the sleep tracking accuracy of the fitbit sense. Specifically, i will check if it can correctly predict the different sleep stages based on data i collected over the last 6 months. With that i mean, can it correctly detect when you’re in deep sleep light sleep and rem sleep? Additionally, i want to check if it can detect when you fall asleep at night and when you wake up in the morning and if you can also detect the moments. I woke up during the night. My channel is not so much about listing features. Instead, on my channel, i try to test the accuracy of different measurements. However, before getting to the test, let me explain the basics on how fitbit tracks your sleep. Specifically, how does your fitbit sense detect your sleep stages? Fitbit estimates, your sleep stages, using a combination of your movement and heart rate patterns. Fitbit says that when you haven’t moved for about an hour, your tracker or watch assumes that you’re asleep. Additionally, they use things such as the length of your movement. You have as an indication of your sleep behavior, for instance, rolling over.
This also helps confirm that you’re asleep, while you’re sleeping your fitbit tracks the changes in the time between heartbeats, which is known as heart rate variability. These can also help estimate your sleep stages as these fluctuate, as you transition between light sleep, deep sleep and rem sleep for the sleep comparison, i tested the fitbit sense for 11 nights. At the same time, i also wore this portable scientific eeg device, and i recorded myself using an infrared camera, the eeg device measures, brainwaves and muscle movements, and is being used by several of my colleagues in scientific studies. I manually went through the recording of the eeg and scored each part of the night for the different sleep stages. I also exported my data from the fitbit servers and wrote some code to extract it and analyze the data with the infrared recording. I can actually check what my movements were like and see if the fitbit sense correctly predicts when i’m awake now let’s review the results i obtained let’s. First, have a look at the accuracy of 5 example knights to see what patterns we observe, after which i will do a statistical overview analysis. Here we see the first night i recorded on top. You see the sleep stages as they were recorded using the eeg device on the horizontal axis. We have the time of night and, as you can see, i went to bed quite late, a little bit after 1 pm on the vertical axis.
You have the different sleep stages, that being deep sleep, light sleep, rem, sleep and awake. Now the sleep stages are plotted in the same order that are usually displayed in research on the bottom. You can see a similar plot, but now for the sleep stages as they were recorded using the fitbit sense. If we first look at deep sleep according to the eeg which are marked here in purple, we see quite a good overlap between the fitbit sense and the eeg device. The third deep sleep segment is slightly longer than the one tracked by the eeg device, but overall the tracking is pretty good. Next, looking at rem, sleep, which i marked here in red, we again see a pretty good match for most of the night. It picked up. Almost perfectly on the first three segments and the beginning of the fourth and final segment. Only during a phase where i was alternating a lot between being awake and rem, sleep did it identified those moments. As light sleep instead of rem, sleep to see the sleep cycles, i added non rem, sleep in blue and again marked rem, sleep in red. Each sleep cycle starts with a combination of deep and light sleep together called non ram and always enim ram. Overall, we can see the sleep cycles pretty well based on just the data from the fitbit. Most of the sleep cycles match almost perfectly with the eeg device, except for the last one.
Next, looking at the awake moments marked here in green, we see that these almost perfectly overlapped with the eeg device. All four moments were detected, though they lasted a bit longer than how i scored them with the eeg device. However, overall, this is quite good. Finally, it was also really good at detecting the moment. I fell asleep this night and the moment i woke up, which i marked here in yellow brown. Now this is the second night. I want to show you looking first at deep sleep, we again see that all the deep sleep i had was indeed detected. However, some extra deep sleep was detected in the beginning and also near the end of the night. Still overall, this is pretty good. Next, looking at rem sleep, we again see a pretty good match only near the end. The last ram sleep segment is slightly shifted to the left, but again the fitbit is pretty accurate. This also means we can see most of the sleep cycles for this knight. My one awake moment was also detected, though some extra awake moment was detected as well. Detecting the moment i fell asleep and woke up were also pretty good, though it did detected me falling asleep, a tiny bit too late. For the last three example nights. I quickly want to show you the most important parts, starting off with deep sleep here. We can see the deep sleep for the third night. We see that the deep sleep detection is not that bad, but it is slightly shifted for the second and third segment.
Additionally, there is some extra deep sleep near the end of the night. This is the deep sleep for the fourth night, and here we see that the fitbit sense is pretty much spot on with detecting deep sleep. Now, looking at rem sleep for the third night, we again see, this is very good. The fitbit sense was able to pick up on all four ram sleep segments, which also means we can nicely see the different sleep cycles. For the fourth knight. We can also see a nice match for ram sleep, picking up on all four segments and again we can also see the different sleep cycles. Finally, i wan na have a look at the awake moments here. We can see that for the third night it picked up on two out of four awake moments. This is interesting because before the fitbit sense seemed to pick up on more awake moments, but this night it actually picked up on fewer. For the fourth night, it detected one out of two awake moments, but it did detect an extra awake moment later on in the night. Now this is the final night. I want to show you now. This is not a typical night, but one of the worst scoring nights of the fitbit here you can see it missed the one awake moment i had, but it also detected a lot of extra awake time during the night. However, again most nights, it was pretty good at detecting the moment.
I fell asleep and also at detecting the moment. I woke up, though, here we can see that for this night it came a bit too soon, overall, i’m, pretty impressed with the sleep tracking of the fitbit sands. As i mentioned in many of my other videos in general, i recommend fitbits for their sleep tracking. However, it’s always nice to see this confirmed when doing detailed, checking on a newer model. The fitbit generally picked up on deep sleep rather well. Additionally, it could detect rem sleep and thereby we could also see the sleep cycles most of the time it picked up on a lot of my awake time, though, it sometimes detected a bit too much of it to get an even more objective view of the results. Let’S calculate some statistics regarding consistency between the sleep stages of the fitbit sense and eeg device. However, first a quick side note if you’re interested in the latest updates on the wearables i’m testing consider subscribing to my instagram and my weekly newsletter. Of course, you would also make me really happy if you subscribe to this youtube channel now enough self promotion, let’s see what the overview statistics say. First, let’s look at the total percentage of each sleep stage, the eeg device and the fitbit sense predicted. Overall, these percentages are a bit off. We see that the fitbit sense predicts a little bit too much deep sleep, which is consistent with what we saw for the individual nights.
It also predicts too much awake time about double the amount. The total amount of rem, sleep and light sleep are slightly lower than they should be more important, even than these total percentages is checking if the sense predicts the correct sleep stages at the right time and that’s. What i displayed here on top, we have the sleep stages according to the eeg device and on the left, the sleep stages according to the fitbit sense. Now each column here sums to 100, meaning that we can see what percentage of each of the actual sleep stages was recorded as each sleep stage by the fitbit sense. If we first look at deep sleep, we see that almost 70 percent of what was deep. Sleep was also correctly predicted as deep sleep by descents. If it was confused, almost all of the deep sleep was confused with light sleep next, looking at light sleep, we see that also about 70 percent of the light sleep is correctly predicted and if it is confused, it is most often confused with deep sleep and also Sometimes, with rem sleep now, rem, sleep is correctly predicted almost 60 of the time, and what we saw in individual nights is that the sense is able to pick up on most of the rem sleep segments, even if they’re slightly too short or slightly shifted. This means that we can see the sleep cycles most of the time. If rem sleep is confused, it’s, most often confused with light sleep.
Finally, awake time is almost 80 percent of the time correctly predicted, which is really good. If it is confused, it is most often confused with light sleep which of course makes sense. So this is all looking really good for the fitbit sense. It tracks most of the sleep stages correctly and is able to detect the sleep cycles i went through. So how does this compare to other fitbit devices in the next section? I’Ll, compare it to the fitbit charge 2 charge 3 and charge 4, and also to just release fitbit lux i’ll. Also compare it in even more detail to the fitbit inspire 2, which is one of the cheapest wearables. That fitbit makes i wore both the fibbit sense and all these other devices. At the same time, for one or more nights, i exported the data for all of these and calculated what percentages of the knight they predicted the same over all sleep stages. That is what i displayed here on the vertical axis. You can see the different devices i tested the fitbit sense against the fitbit charge 2 against two different bit charge 3s, also against two different fiber charge, fours and against the fibo inspire 2 and the new fibbit lux that was recently released. The percentage of each of these squares is the average percentage over each of the knights that these different fitbits and the fitbit sends agreed about which sleep stage i was in, and the number between brackets is the number of nights i wore both devices.
At the same time, as you can see, the agreement is always roughly between eighty percent and eighty seven percent, which is pretty good based on slight variations in movement and heart rate detection. The algorithm has some variants, but for the most part, there’s a decent agreement between the different fitbit devices. You might remember that when i tested a consistency between two aura rings, the best agreement between any of the sleep stages was about 70 percent, and the overall agreement between the two rings was less than 67 percent, which is much lower than what we see for the Fitbits, this indicates that the fitbit algorithm is more stable than the ordering algorithm when it comes to scoring the different sleep stages. Next, i want to compare the accuracy of the fitbit sense to that of the fitbit inspire 2 in more detail. I think this is interesting, since these devices are basically on different sides of the fitbit spectrum. When it comes to price, you can get the fitbit inspire 2 for around 70 to 80 bucks, whereas the fitbit sense generally is priced at around 250 to 300. Here i displayed the performance of the fitbit sense on the left and the inspire 2 on the right both compared to the eeg device. As you can see both performed about the same, they can detect the most awake moments and the other sleep stages are correctly predicted. Almost 70 percent of the time, both of them show most difficulty detecting rem, sleep correctly, but i would still say both of them are rather good.
Overall, these results confirm for me that all fitbit devices i’ve tested so far perform really well when it comes to sleep, tracking and no single fiber device, outperforms all the others, so you can basically pick any device from the lineup and you will get good performance. The final thing i want to check in more detail is how accurately the fitbit sense can detect the moment i wake up and the moment i fall asleep, and that is what i plotted here on the vertical axis. We have the dates of the night that i tested the sense and on the horizontal axis, is the time difference between the eeg device and fitbit sense for waking up in yellow and falling asleep in blue now, a positive number means that it detected me waking up Or falling asleep later than in reality, and a negative number means that the fitbit sense says i woke up or fell asleep earlier than i actually did. As you can see, the largest difference over any of these nights is about 15 minutes, but in general the difference is just a few minutes maximum. The only thing i can say is that in general, it tends to detect me falling asleep slightly too late. Since all the blue points on the right side of the line, however, this is always just by a few minutes, so this is still really accurate overall, i’m, very satisfied with the sleep tracking accuracy of the fitbit sense.
It predicts the correct sleep stages at the right time for the majority of the night. Additionally, it can detect the moment i wake up and fall asleep quite well, and it also allows you to see your sleep cycles. The performance overall appears to be about the same as for any other fitbit device, which is really good. So should you buy the fitbit sense? Well, if you’re interested in a smartwatch with good sleep tracking, then i can definitely recommend the fitbit sense. Fitbit devices have always been the best devices i’ve tested so far for sleep tracking, and this is no different for the sense. The sense is fitbit’s flagship, smartwatch and therefore also comes in at a relatively high price if you’re not interested in any of the features specific to the sense. Like the smartwatch features, the eda sensor for stress, detection or ecg sensor for arrhythmia detection, then you could also consider buying a cheaper device like the fitbit, inspire 2 or the fibo charge. 4 i’m currently also testing the fitbit lux and the first night i tested it. It also seemed pretty good. You should also consider that, when the sense originally launched it did have some heart rate detection issues during workouts i’m still testing, whether or not this has been solved due to software updates. So this is something to consider when buying this sense. Finally, i should mention some of the limitations of the data that i showed here. First of all, i just tested the watch on me and just for a limited number of days and it’ll be interesting to see how it performs on others to do a full sleep.
Comparison, it would be good to also test the watch against the full scientific polysonography setup. We actually assembled the polysonography device using openvci components and we’re now working to get the software functional. This way, i will not have to rely on sleep labs for my testing, which is especially difficult in these times of corona. Again, a big shout out to my colleague rob for the 3d printing and putting it together and my colleague freddie for the hardware and software expertise in my videos. I do scientific tests on different devices like the aura ring the fitbit and the scan watch and, in the end, i hope to use tracking to improve my life. So if you like that, subject and like this video consider subscribing to my channel and also consider giving it a thumbs up, because it makes it easier for other people to find my videos. Thank you so much for watching and also consider watching.IqtkUoVcpFs
