# YouQuantified in the Classroom ## Overview {% embed url="" %} ### Platform **YouQuantified** gives learners the opportunity to explore data generated by their own brains, bodies, and behaviors. With the support of hands-on lessons that introduce them to basic principles of data, data visualization, and data ethics, users will work with biosensing devices and web-based creative coding tools to develop their own representations of their brainwaves, heart rate, voice, etc. They pursue their own questions about the data they generate, contextualize their observations within population-wide patterns, and discover ways in which data plays a role across a range of STEM careers by connecting with professionals who work with human data. {% hint style="info" %} Currently, the app has restricted access. You can join the waitlist at [youquantified.app](https://youquantified.app/) or reach out directly to us via email at {% endhint %} ### Curriculum You can find more information in the [educational resources](/educational-resources) section of this guide. ## Platform usage If this is your first time visiting the platform, please check out our [quick start guide](/quick-start). {% content-ref url="/pages/oKAONB0bGxckcUVSy3Wr" %} [Quick Start](/quick-start) {% endcontent-ref %} ### Summary of devices Below, we provide a summary of the devices that may be used with the web application. Some of them may require the purchase of separate hardware while others may be used with your computer's webcam or microphone. You can find more information under the [supported devices](/devices/file-upload) tab.
DeviceWhat does it stream?Pre-requirements
File uploadPre-recorded dataA previous recording with the app.File Upload
Face LandmarksFacial expression metricsCameraFace Landmarks
Video Heart RateHeart rateCameraVideo Heart Rate
Audio VolumeAmbient noise levelsMicrophone
Muse

EEG & bandpower

PPG & heart rate

Muse 2 hardwareMuse
EMOTIVEEG & band powerEMOTIV-hardwareEMOTIV
LSLMultiple types of streamsLSL-Compatible Scientific Equipment
## Contributing to development This app was built as an open-source platform. For project maintenance and documentation, you may find some information in the developer's section of this guide and check out the full code powering the web application at the following GitHub. {% embed url="" %} As an open-source platform, this app leverages other technologies, such as [P5.js](https://p5js.org). You may find more information as you dive into this documentation. ### About Us & Credits This app was developed at NYU through a collaboration of multiple research laboratories as a project aimed at promoting students’ data literacy through their creative representation of the Quantified Self. Our team is also responsible for the development, maintenance, and implementation of the [MindHive platform](https://mindhive.science). This project is led by [Suzanne Dikker](http://www.suzannedikker.net), [Camillia Matuk](https://wp.nyu.edu/riddle/), and [Xavier Ochoa](https://steinhardt.nyu.edu/people/xavier-ochoa) at New York University with support from the National Science Foundation. # Educational Resources A MindHive Program @ New York University ## Program Overview ***YouQuantified*** gives middle- and high school students the opportunity to explore data generated by their own brains, bodies, and behavior. With the support of hands-on lessons that introduce them to basic principles of data, creative data visualization, and data ethics, learners will work with biosensing devices and [web-based creative coding tools](https://youquantified.com/) to develop their own representations of their brainwaves, heart rate, sweat, movement, voice, etc. They will pursue their own research questions and curiosity about the data they generate, contextualize their observations within population-wide patterns, and discover ways in which data plays a role across a range of STEM/STEAM careers by connecting with professionals who work with human data. **Details** * Funding: [US National Science Foundation’s Innovative Technology Experiences for Students and Teachers (ITEST) program](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2241751\&HistoricalAwards=false), * Lead institution and lead investigators: [Dr. Dikker](https://www.suzannedikker.net/), [Dr. Matuk](https://wp.nyu.edu/riddle/), and [Dr. Ochoa](https://steinhardt.nyu.edu/people/xavier-ochoa) at New York University. * Age range: adaptable for both middle school and high-school aged children * Format: adaptable for both formal and informal contexts (more below) **Requirements** * **Let us know that you are planning on using our tools.** Please note that if you plan to use any of our tools in your classroom, we do require that you set up an introductory meeting with our team. It is critical for us to know who is using You Quantified to (a) provide appropriate support and (b) for reporting purposes to our funders. * **Participate in our research & program evaluation.** You will also be asked to participate in our educational research and program evaluation activities. While participation takes place on a voluntary basis, program evaluation is critical to keeping our project alive. {% hint style="info" %} **Contact us** at for more information or to set up a call. {% endhint %} ## Program Tracks We currently offer four YouQuantified program tracks. Each has its own set of requirements. If you are interested in using YouQuantified in your course, please reach out to to discuss which track is best for you and which level of support you will need (see Support). Knowing who is using You Quantified is critical to keeping the project alive. {% tabs %} {% tab title="1. Data Explorer Track" %} In the role of Data Explorer, students will construct a personal data biography /data autobiography (i.e. stories of how a dataset was produced and used). Students will engage in a series of activities in which they explore their identity through interpretation of brain, body, and behavioral data visualizations. **Recommended class time:** 2-4 class periods **Recommended support:** Level 1-2 {% hint style="info" %} Example coming soon … {% endhint %} {% endtab %} {% tab title="2. Data Maker Track" %} In the role of Data Maker, students will develop either a creative data visualization of their own data or a data-driven application that others can use to visualize personal data. Students will explore the multiple purposes and strategies for data visualization and how digital applications draw on personal data from users. **Recommended class time:** 6-8 class periods **Recommended support:** Level 2-3 {% hint style="info" %} Example coming soon … {% endhint %} {% endtab %} {% tab title="3. Data Ambassador Track" %} In the role of Data Ambassador, students will produce a data visualization and add a statement to their data biography that explores ethical decision-making around how data is collected and used. **Recommended class time:** 6-8 class periods **Recommended support:** Level 2-3 {% hint style="info" %} Example coming soon … {% endhint %} {% endtab %} {% tab title="4. Data Scientist Track" %} In the role of Data Scientist, students will propose and run an experimental research study. Students will engage in a process of scientific inquiry and learn about experimental design. **Recommended class time:** 10-12 class periods **Recommended support:** Level 3 {% hint style="info" %} Example coming soon … {% endhint %} {% endtab %} {% endtabs %} ## Curriculum Materials ***YouQuantified*** comprises 10 modules that can be flexibly adapted as relevant and appropriate. Each module discusses a separate data source (e.g., brain, heart rate, voice, video, movement) in the context of meaningful exploration (e.g., attention in the brain, heart rate variability and emotions).
{% hint style="warning" %} **Access to curriculum materials will be provided upon registration with our team at** {% endhint %} #### Each of our modules is divided into four sections: Explore, Learn, Create, Apply. See sample slides from our program curriculum. * **Explore**: Students are guided through exploring a given biosensing device, and are encouraged to think about what they notice and observe about the data they generate during this process. * **Learn**: Lesson content is delivered, contextualizing student observations that might have emerged from their explorations. * **Create**: Students engage in an exploration. With a given “exploration question”, students conduct a mini-experiment by collecting data and developing a visualization of this data – hand-drawn, or through our [You Quantified web-based app](https://creative-quantified-self.netlify.app/visuals/default). * **Apply**: Students have the opportunity to connect with industry partners who work with datastreams discussed in the course, and to access career and college readiness resources relevant to these fields.
### Tools **YouQuantified App** The You: Quantified web app is publicly available. You can explore the platform and stream video and voice data. Authenticated users can also create their own projects and stream prerecorded brain and body data. \ **MindHive Platform**\ MindHive is a platform designed to support teachers and students in carrying out independent projects. Within the platform, teachers and students can engage with tools, mentorship opportunities, and educational resources in ways that are designed to support multiple forms of participation with personal data. ***Conditions of use:*** While there are no restrictions in using the *YouQuantified* app or MindHive Platform, we ask that you contact us if you plan to use it for educational purposes (see above). ### Services The MindHive team provides three levels of support for instructors who are interested in using the You:Quantified curriculum and tools. Level 1 participants agree to attend a virtual onboarding session and have the option to attend professional development and weekly drop-in meetings throughout the semester. Levels 2 and 3 participants are required to attend professional development meetings and weekly drop-in meetings throughout the semester. Please note that only level 3 participants will have access to our technology loan program.
**1. Professional Development (PD)** We offer support through professional development and weekly office hours. Under some conditions, stipends and/or professional development credits can be made available. ***Conditions of use:*** Participating in professional development is a requirement for teachers who are interested in using biosensors in their classroom and for teachers who choose the Data Scientist Track. Contact us at **2. Biosensors (limited capacity)** We offer a biosensor loan program for teachers who agree to participate in a professional development program and receive in-class support from mentors. Teachers who participate in this program will have access to biosensing devices, including headsets that measure brainwaves (EMOTIV and Muse) and physiology (Emotibit). ***Conditions of use:*** If you plan to use biosensors in combination with YouQuantified, contact us at . Loans and support will only be provided to teachers who participate in our professional development and mentorship program. **3. Mentor Support (limited capacity)** We offer live support by professional neuroscientists who can help your students with their projects. ***Conditions of use***: Mentor support is a requirement for teachers who use biosensors in their classroom, and highly recommended for teachers who participate in the Data Scientists Track. Mentor support is conditional upon participation in our professional development sessions. # Quick Start Start using our application {% hint style="info" %} **Before starting,** if you are a student or want a refresher on biosensing and data, we recommend that you take a look at the [educational resources](/educational-resources). {% endhint %} This section will teach you how to use the application and guide you through the process of: 1. [#connecting-to-a-device](#connecting-to-a-device "mention") 1. [#using-pre-recorded-data](#using-pre-recorded-data "mention") 2. [#using-your-computer](#using-your-computer "mention") 3. [#other-devices](#other-devices "mention") 2. [#recording-data](#recording-data "mention") 3. [#working-with-a-visual](#working-with-a-visual "mention") 4. [#code-editing](#code-editing "mention") 1. [#making-an-account](#making-an-account "mention") 2. [#changing-a-visual](#changing-a-visual "mention") ## Connecting to a device 1. The first step when connecting to a device is to go into the "devices" window at the corner. Here you can view every available data source. 2. To **add a new one**, click on the card and connection will be attempted. Below is a list of suggestions on the different types of devices available within the platform for you to get started. Once you have data streaming in the platform, you may[ record the data ](#recording-data)or [head directly into a visual](#working-with-a-visual) ### Using pre-recorded data 1. To upload, add a new device, select *Upload a file* and a new popup will appear 2. Here, you must upload the original zip file used for the recording. You may also make small changes and compress the files into a new zip, but the web app may fail to reproduce those files. The object that is being played back should behave similarly to a connected device. You need to click on the **play button** to start streaming the data. Once the file ends, playback will stop, but you have the option to start again, rewind, or loop. ### Using your computer Some devices don't require additional hardware and may work with your computer's webcam and microphone, such as the [Face Landmarks](/devices/face-landmarks), [Video Heart Rate](/devices/video-heart-rate), and Audio Volume. To connect make sure to: * Always leave the window open in the same browser. Closing it or dismissing it may interrupt the stream. * Grant browser permission. Whatever data you record will remain local and will never be sent to any of our servers. ### Other devices Other devices may require specialized hardware and slightly different instructions. Please visit the sections for the two EEG & PPG devices for which we currently offer full support. * [EMOTIV](/devices/emotiv) * [Muse](/devices/muse)
For scientists – LSL You may also connect local LSL datastreams to the app; however, this functionality doesn't have full support. For more information [view the respective page in the guide.](/devices/lsl)
## Recording data 1. To record a file, simply start streaming using your device and press the **record button,** accessible at any time from the "devices" window. 2. It will save the streamed data to your computer as a .zip with multiple .csv files that contain data points, timestamps, and some metadata. {% hint style="danger" %} Avoid **deletion** of the .zip file if you ever want to play back the pre-recorded data. {% endhint %} ## Working with a visual The "visuals" tab provides the list of visuals that are available in the platform. If it’s your first time using the tool, we recommend following the visualizations on the home page. It helps to learn how data is mapped into the application, how to adjust the range, and some of the metrics we’re getting. Once you have selected a visual: 1. In the left window, you can control parameters manually or assign biometric data to the field using the toggles at the edge of the parameters. 2. You can adjust the range manually by looking at the numeric values or choosing auto range. This function waits until it has acquired a certain amount of data points. It then calculates the minimum and maximum from that buffer and maps the value so that it falls within that range. #### Suggested starting visualizations
Circle tutorialThe circle tutorial is the best visualization to learn about code editing and customization.
Power barsThe power bars are an amazing visualization to show the real-time changing values of each frequency band. They also include a time-series plot.
## Code editing All visuals are created based on P5.js, so you should be able to follow [any of their tutorials](https://p5js.org/tutorials/) to learn about its usage. This guide will cover platform-specific materials. ### Making an account You can use the app without making an account and access most of its functionality. Code editing is the only feature limited to registered users. Other features may come soon for registered users. If you want to create an account follow the instructions 1. Go to the [youquantified.com ](https://youquantified.com)URL and navigate to "explore visuals" 2. You can log in or sign up with your email, a unique username, and a secure password. ### Changing a visual After you have created an account on the platform, code editing allows you to edit any visualization, delete parameters, and add new ones. 1. To get started, simply click on the edit button on any of the ones we have. A copy will be created. * In this screen, you can rename your visualization by going into the top bar (when you access it again you won’t be able to rename it, be careful). 2. To edit the code, use the code icon. Any changes will be instantly made live. You can download your code as text at any time. 3. To add or delete parameters, simply click the + or - icons. New parameters can only have unique names. We recommend using simple names. To reference parameters in code, you must use the following syntax: ```javascript data?.[“MY PARAMETER NAME”] ``` Where `“MY PARAMETER NAME”` is the exact name of your parameter as a string. # Add a new visual The basics of adding a new visual are already covered in the [quick start guide](/quick-start) under [code-editing](/quick-start#code-editing). This section will dive into the specifics of how visuals work on the backend and other advanced features. ### Uploading files when creating visuals When creating a new visual, users have the option to upload a file. You may upload a `.js` or .`txt` file that was directly downloaded from [P5.js](https://p5js.org) or created using the You: Quantified web app. ### Having a featured visual To have a visual published under the main page, it must be approved by an admin of the platform. Feel free to reach out to us in case you have something interesting. ### Other types of visuals Due to the way in which the platform handles visuals (as a code sandbox), it's technically possible to embed any type of JavaScript or React content in that iframe. Feel free to reach out to us if you have an interesting creation, such as a WebGL videogame. # Create a new device # Event markers While You: Quantified mainly focuses on online visualization and real-time experiences, event markers also exist if someone wants to use them for recordings. Event markers are treated like other devices but are streamed from within the visualization. This means you trigger event markers from within a visual. ### Function to send event markers To send event markers, call the following function inside a P5.js visual. ```javascript sendEvent(message); ``` The message has to be a JavaScript object, which means it needs to have a key, and value pair structure, such as: ```javascript const message = { key: value } ``` This also means you can do something more akin to a JSON. ```javascript const message = { key1: value1, key2: value2, key3: value3, } ``` With that pre-scripted function, you will send an event that will be **saved alongside the timestamp** when it was transmitted. While an advanced use-case, by leveraging P5.js and JavaScript function, you can send events triggered by * [A key press](https://p5js.org/reference/p5/keyPressed/) * [A button press](https://p5js.org/reference/p5/createButton/) * [A specific time interval](https://developer.mozilla.org/en-US/docs/Web/API/setInterval)
Look into the code behind the function Event markers were added through the [postMessage method](https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage) provided in JavaScript. They are a function being called in the P5.js iframe, which opens up the possibility for integration into other platforms. ```javascript function sendEvent(message) { if (typeof message === 'object') { window.parent.postMessage(JSON.stringify(message)); } } ```
# MIDI Signals Get MIDI output from You: Quantified devices. For people interested in performative arts, there are easy workarounds to output You: Quantified signals as MIDI by leveraging the capabilities of the [Web MIDI API](https://developer.mozilla.org/en-US/docs/Web/API/MIDIOutput). A quick implementation is shown in the beta version of the [MIDI controller visualization](https://youquantified.com/visuals/clwo77ykc0005ptjvtuil8vjb). It is also important to be familiar with the MIDI output standard to know how to send notes and messages. ### MIDI Protocol When sending MIDI signals, you (as the programmer) are engineering a new MIDI device! It is exciting, but it also means there is a bit of a learning curve. The MIDI Association has a rather comprehensive [article](https://midi.org/about-midi-part-3midi-messages) explaining the notes and a useful [table](https://midi.org/expanded-midi-1-0-messages-list) that summarises some of that information. For this guide, I will introduce basic concepts that I find the most useful and cover them in a way that's intuitive to start with. However, by all means, feel free to dive deeper and discover the depths of this protocol. First, messages are usually output as a series of 3 data bytes (status + 2 bytes of data). This would make an array with three numbers: ``` message = [status, data1, data2] ``` You can write the data in binary, decimal or hex format (the table will often contain every one of those representations). #### Control Change Control change messages are meant to be used for knobs or pedals. The message you send depends on the rotation of your knob. Here, the three bytes represent: 1. Channel – There are multiple channels that range from 176 to 191 that may be used for different functions. 2. Control Change – This would be the equivalent of specifying a "knob". Each channel can have 127 different knobs. MIDI controllers sometimes reserve certain numbers for specific functions (i.e. a pedal vs an actual knob), however, that doesn't necessarily have to be the case. 3. Value – The value that you're sending, ranging from 0 to 127. #### Notes These messages are meant to be used for playing keys or notes. A note message can be broken down into the following components: 1. Channel & Status – There are a total of 16 channels (think of them as 16 different piano keyboards). However, the first byte changes depending on whether you are sending a channel-on or channel-off signal. For example, channel 1 has a hex value of `0x80` for note off and `0x90` note on. 2. Note – Next comes the actual key you are playing. Different piano keys are represented by different numbers. The middle C, for example, has a note value of 60. There are a lot of [images](https://www.tinyloops.com/doc/drumcomputers/) that compare them to pianos. They range from 0 to 127. 3. Velocity – This last number would represent how hard you pressed the key on a scale from 0 to 127. ### Preparing a virtual MIDI port on my computer The MIDI output from the web requires a virtual device to work. This means you have to create an internal device that's unoccupied and running in the background. Depending on your operating system, the steps to follow are slightly different. Outlined below are some recommended tutorials: * [MacOS](https://medium.com/@keybaudio/virtual-midi-devices-on-macos-a45cdbdffdaf) * [Windows](https://manuals.morningstar.io/mc-midi-controller/creating-virtual-midi-ports-in-windows-os) ### How to integrate MIDI into P5.js code? The general steps followed for the visual will be outlined here so that users can integrate MIDI output into any visual or adapt the code to their liking. #### Step 1 - Create global MIDI variables to store the object ``` let midi = null; let outputPort = null; ``` #### Step 2 - Create functions that may act as either buttons or knobs. For noteOn/noteOff messages (which act like keys from a piano), a simple function can look like the following: ```javascript function updateMIDINote(midiOutput, noteOn, note_num) { let noteStatus = noteOn ? 0x90 : 0x80; // If note is off, turn it on and viceversa const noteOnMessage = [noteStatus, 60 + note_num, 127]; // note on, full velocity midiOutput.send(noteOnMessage); // sends the message. } ``` Providing an example for a control change message taken from a value that ranges between 0 and 1. ```javascript function controlChangeMessage(midiOutput, controlNum, value) { let controlChangeMessage = [0xB0, controlNum, floor(map(value, 0, 1, 0, 127, true))]; midiOutput.send(controlChangeMessage); } ``` #### Step 3 - Integrate functions into your code It's important to be mindful of the fact that a void draw gets called every time a frame is refreshed in P5.js, so handling MIDI logic on your draw function may be tricky. There are multiple multiple approaches: {% hint style="success" %} Handling everything in the draw function and making additional code to save previous values. You can take a look at [this](https://youquantified.com/visuals/cm1gso9g20007ptbyprk8wl2y) example. {% endhint %} {% hint style="info" %} Overwriting the data event listener function and adding the MIDI logic in that function. ```javascript window.addEventListener("message", (event)=>{ data = JSON.parse(event.data); // Add MIDI logic that works with the data in here }) ``` {% endhint %} # File Upload ## Uploading data 1. To upload, add a new device, select *Upload a file* and a new popup will appear 2. Here, you must upload the original zip file used for the recording. You may also make small changes and compress the files into a new zip, but the web app may fail to reproduce those files. The object that is being played back should behave similarly to a connected device. You need to click on the **play button** to start streaming the data. Once the file ends, playback will stop, but you have the option to start again, rewind, or loop. ## Recording data 1. To record a file, simply start streaming using your device and press the **record button,** accessible at any time from the "devices" window. 2. It will save the streamed data to your computer as a .zip with multiple .csv files that contain data points, timestamps, and some metadata. {% hint style="danger" %} Avoid **deletion** of the .zip file if you ever want to play back the pre-recorded data. {% endhint %} ## Information about pre-recorded data The files recorded for this application will typically follow the convention `deviceName_datetime`. They will be downloaded as a zip file with the following minimum contents: * README.txt with basic information about the folder and its contents. * metadata.csv where the devices and files are described and correlated The other files will be .csv recordings of the datastreams. If you ever want to unzip a folder, modify a file, compress it, and re-upload it, it is important to avoid modifying the files above, otherwise it might not work. # Muse EEG & PPG Headset The Muse S and Muse 2 are research-grade EEG headbands for meditation and sleep. Visit the [Muse website](https://choosemuse.com) to learn more. ## Using it in the platform The Muse 2 headband is only compatible with Chromium-based browsers, which include Google Chrome, Microsoft Edge, and Opera. **To connect**, you don’t have to download anything. Simply turn on the device and proceed to connect it through the application using Bluetooth. If you have connectivity issues, oftentimes the best solution is to restart the device. ## Signals ### Sensors * **EEG:** electroencephalography measures **brain activity** by capturing electrical activity from the brain. This device has four EEG electrodes meant to capture information from the frontal and temporal lobes of the brain as measured through the scalp. * **PPG:** photopletismography measures changes in blood flow through light absorption or scattering by the hemoglobin in the blood. ### Output #### **EEG raw signals** According to the 10-20 international system, the Muse has the following electrodes. * TP9 * TP10 * AF8 * AF7 #### **EEG band power** Band power quantifies a signal's strength in a specific frequency range. The value provided is the average power across all electrodes. The calculation is performed using a fast Fourier Transform at 20 Hz on a 2-second window * Theta: 4 - 8 Hz band power * Alpha: 8 - 12 Hz band power * Low beta: 12 - 16 Hz band power * High beta: 16 - 25 Hz band power * Gamma: 25 - 45 Hz band power #### **Other** * PPG: raw signal obtained by the light sensor * Heart rate: heart rate calculation from PPG ## Attribution To connect to the Muse on the browser, we used the implementation provided on the following GitHub. {% embed url="" %} # EMOTIV EEG Headset EMOTIV headsets are a lineup of research-grade EEG equipment for brain-computer interfaces and scientific research. To learn more, you can visit their [website](https://www.emotiv.com/products/epoc-x). By installing their custom launcher, this app can connect to any one of the headsets and receive EEG or movement signals. ## Using it with the app Before starting, make sure to charge the device, and pour saline solution (it can just be water with salt) on the electrodes to make them conductive. Afterwards, you can plug in the USB dongle provided to get started. 1. Download the [EMOTIV launcher](https://www.emotiv.com/emotiv-launcher/). Follow the steps described in the app **to connect.** Once the device is turned on and connected, proceed to the web application. 2. Open the devices window of the web application and click on the EMOTIV card to attempt connection. If you're logged in and the device is connected, you may go back to the EMOTIV launcher and get a prompt to approve the connection. 3. The web app should now be able to connect. If you run into any issues, re-attempt it and make sure the device is showing up in the launcher and there are no other permission prompts. {% hint style="info" %} Regardless of whether you have a license or not, you can still connect your device to this app, however, if you don’t have a license you won’t be able to see or use the raw data. {% endhint %} #### Resources * [Connecting your EEG Headset](https://docs.google.com/document/d/16AQA_SosiXh-t3TxD4xfYg7LHqXBWzvemVvP0wAE29k/edit?usp=sharing). * [EMOTIV](https://www.emotiv.com) ## Signals ### Sensors * **EEG:** electroencephalography measures **brain activity** by capturing electrical activity from the brain. This device has four EEG electrodes meant to capture information from the frontal and temporal lobes of the brain as measured through the scalp. * **Accelerometer & Gyroscope:** the accelerometer and gyroscope provide information about the movement and rotation of the device in 3D space. ### Output Since the launcher provides an interface for every EMOTIV product, different headsets might have different EEG electrodes. All EMOTIV systems are aligned and described their positions according to the international 10-20 naming system. #### EEG raw signal With a paid subscription, the EMOTIV will provide raw values for each electrode. Using the EMOTIV X (one of their flagship products) as an example, you would get the following electrode locations * AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4 #### EEG band power Band power quantifies a signal's strength in a specific frequency range. The calculation is performed by EMOTIV according to their [specifications](https://www.emotiv.com/tools/knowledge-base/research-information/how-are-band-powers-calculated?srsltid=AfmBOool5hV84faMH3hpZkTfRWeOJYMf280E6RJSXzJCkUAhw9jxjklg) and averaged across all electrodes. * Theta: 4 - 8 Hz band power * Alpha: 8 - 12 Hz band power * Low beta: 12 - 16 Hz band power * High beta: 16 - 25 Hz band power * Gamma: 25 - 45 Hz band power # EmotiBit PPG, EDA, Temperature & Motion {% hint style="info" %} We are working to add EmotiBit support to the YouQuantified platform. In the meantime, you can use it with their software. {% endhint %} ## Working With Data On Their Software You can record and visualize your physiological and motion data using the free and open-source EmotiBit software. Follow [their documentation](mailto:undefined) for a guide on how to get started with using it. #### Resources * [Official EmotiBit Documentation](https://github.com/EmotiBit/EmotiBit_Docs/tree/master) * [Preparing the EmotiBit for In-Class Usage](https://docs.google.com/document/d/17qMcnzGUGvR_d6HZP0mW9HH8pLts805u2XTwn9Ke6lU/edit?tab=t.0#heading=h.ca3yxly2ocm8) * [Using the EmotiBit in the Classroom](https://docs.google.com/document/d/1Zhkjt6K3NbiQoGj7yltwn811aC5uLyJIAG5pRYvWj_U/edit?tab=t.0#heading=h.ca3yxly2ocm8) # Face Landmarks Video-based Facial Expression Detection Face landmark detection relies on the open-source project, [Media Pipe](https://ai.google.dev/edge/mediapipe/solutions/guide). They provide AI/ML models that run on-device to analyze live video frames. This feature of the platform uses face landmark detection to calculate scores (coefficients corresponding to different expressions) of 52 locations across the face.
More information about the model Traditionally, these models have been used to get face locations, calculate expressions, and, most notably, apply effects and filters.
### Output The following list provides the predicted shapes ``` 1 - browDownLeft 2 - browDownRight 3 - browInnerUp 4 - browOuterUpLe 5 - browOuterUpRight 6 - cheekPuff 7 - cheekSquintLe 8 - cheekSquintRight 9 - eyeBlinkLe 10 - eyeBlinkRight 11 - eyeLookDownLe 12 - eyeLookDownRight 13 - eyeLookInLe 14 - eyeLookInRight 15 - eyeLookOutLe 16 - eyeLookOutRight 17 - eyeLookUpLe 18 - eyeLookUpRight 19 - eyeSquintLe 20 - eyeSquintRight 21 - eyeWideLeft 22 - eyeWideRight 23 - jawForward 24 - jawLeft 25 - jawOpen 26 - jawRight 27 - mouthClose 28 - mouthDimpleLe 29 - mouthDimpleRight 30 - mouthFrownLe 31 - mouthFrownRight 32 - mouthFunnel 33 - mouthLe 34 - mouthLowerDownLe 35 - mouthLowerDownRight 36 - mouthPressLe 37 - mouthPressRight 38 - mouthPucker 39 - mouthRight 40 - mouthRollLower 41 - mouthRollUpper 42 - mouthShrugLower 43 - mouthShrugUpper 44 - mouthSmileLe 45 - mouthSmileRight 46 - mouthStretchLe 47 - mouthStretchRight 48 - mouthUpperUpLe 49 - mouthUpperUpRight 50 - noseSneerLe 51 - noseSneerRight 52 - tongueOut ``` This information was obtained from their [documentation](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker/index). # Pose Landmarks Video-based Pose Tracking Pose landmark detection relies on the open-source project, [Media Pipe](https://ai.google.dev/edge/mediapipe/solutions/guide). They provide AI/ML models that run on-device to analyze images or live video frames. This feature of the platform uses [pose landmark detection ](https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker)to detect people in the scene and track the approximate location of 33 body parts. ### Output The models output each landmark's normalized location of x, y, and z coordinates. The three coordinates correspond to: * `x` and `y`: Landmark coordinates normalized to `[0.0, 1.0]` by the image width and height respectively. * `z`: Represents the landmark depth with the depth at the midpoint of hips being the origin, and the smaller the value the closer the landmark is to the camera. The magnitude of `z` uses roughly the same scale as `x`. The following list describes the approximate key points they track: ``` 0 - nose 1 - left eye (inner) 2 - left eye 3 - left eye (outer) 4 - right eye (inner) 5 - right eye 6 - right eye (outer) 7 - left ear 8 - right ear 9 - mouth (left) 10 - mouth (right) 11 - left shoulder 12 - right shoulder 13 - left elbow 14 - right elbow 15 - left wrist 16 - right wrist 17 - left pinky 18 - right pinky 19 - left index 20 - right index 21 - left thumb 22 - right thumb 23 - left hip 24 - right hip 25 - left knee 26 - right knee 27 - left ankle 28 - right ankle 29 - left heel 30 - right heel 31 - left foot index 32 - right foot index ``` This information was directly retrieved from their [documentation](https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/pose.md#output). # Video Heart Rate Video-based Heart Rate Estimation To estimate the heart rate from video we are using an open-source algorithm that masks the face from the video using ML models and uses image-processing methods to track subtle changes in color and try to infer a person's heart rate. This method is often known as rPPG (remote photopletismography). {% hint style="info" %} Video heart rate can be greatly influenced by camera quality and lighting conditions. It's best to avoid over or under-exposing the frame. {% endhint %} ### Output * Heart Rate: a single number representing the estimated heart rate. ## Attribution {% embed url="" %} # LSL # Voice Emotion This data source uses a [Speech Emotion Analyzer](https://github.com/tabahi/WebSpeechAnalyzer?tab=readme-ov-file) demo based on the **formantanalyzer** JavaScript library. This algorithm extracts syllables from the Mel-spectrogram of the input and feeds relevant statistical features into a single-layer neural network that classifies. This model was trained on datasets of actors going through dialogues or improvising. It predicts the following categories: * Happiness * Sadness * Anger * Neutral From these labels, we output **valence** and **arousal** scores by using the relative happiness probability (for valence) and neutral probability (for arousal). ## Attribution {% embed url="" %}