TESTING

This guide is provided for SVALT reviewers and those interested in learning how to conduct accurate and repeatable supplemental cooling tests. This guide is what SVALT uses to conduct performance testing as seen on the Performance page. The guide is broken down into the following sections:

• Introduction to Testing
• Testing Types
• Testing Variables
• Testing Data
• Testing Process

Introduction to Testing

Conventional benchmark tests typically gauge performance by testing for a few minutes and then comparing test results to a database of previously conducted tests. The process of testing supplemental cooling differs from typical performance benchmark testing in the following ways:

#1 Repeatability
Accurate measurement of supplemental cooling is achieved by conducting two tests with workloads and conditions that are as close to identical as possible. One test is conducted without supplemental cooling as the CONTROL test, which is the test that determines the laptop's built-in cooling system capacity. Another test is conducted with supplemental cooling as the SVALT test, which is the test that determines the impact of supplemental cooling. The Variables, Data and Process sections cover this topic in greater detail.

#2 Throttling
Processor throttling has a dramatic impact on laptop temperatures and performance. If not familiar with throttling, then please read the Throttling page. Tests conducted to measure temperature should avoid processor throttling, while tests conducted to measure performance require throttling to be present in at least the CONTROL test. The Types section covers this topic in greater detail.

Back to top

Testing Types

As described in the Laptop Throttling page, temperature and performance are directly connected and so cannot be easily tested and measured at the same moment in time. Depending on the testing workload and methods used, temperature and performance can either be tested and measured in separate test sessions or within the same test that divides the test results into pre-throttling and post-throttling data sets.

Temperature Testing
Supplemental cooling increases the laptop system's total cooling capacity, which allows the system to run processors at higher power for a longer period of time with little to no throttling. While it may be counterintuitive, the reality is that effective supplemental cooling that reduces throttling can lead to more heat generation and higher processor temperatures because the processors are using more power to run faster and do more work. This means that temperatures in the SVALT test can be as high or higher than the CONTROL test when throttling occurs in the CONTROL test. Without throttling in the CONTROL test, then the SVALT test will have lower temperatures.

If separate tests are used to measure temperatures, then the testing should apply a workload sufficient to generate heat but not so high to induce throttling. Possible tests might include playing a video downloaded to the local storage drive for one to two hours.

If the same test is used to measure temperature and performance, then the testing data will need to be divided into separate data sets, such as 1) a pre-throttling data set, 2) a post-throttling data set with an increasing rate of throttling, and 3) a post-throttling data set with a steady rate of throttling. Each of the data sets can be tagged with start times in the CONTROL test and then synced up with the SVALT test data to allow for accurate comparisons. At that point each data set can be analyzed for peaks and averages, with the first pre-throttling data set used for temperature comparisons and the third post-throttling-steady-rate data set used for power, speed and performance comparisons. The second post-throttling-rising-rate data set cannot be easily used for comparisons.

Performance Testing
Supplemental cooling helps to reduce heat buildup, however, it can only improve performance when the CONTROL test has enough heat buildup to cause the system to throttle processor power and in turn cause a reduction in temperatures, speed and performance. So to measure performance, throttling must occur in at least the CONTROL test. The time required to trigger throttling can vary from a few minutes to more than an hour, depending on the laptop’s built-in cooling system capacity relative to the laptop’s performance capabilities, the applied workload intensity, and the ambient temperature. In most cases an hour long test is recommended to see how the systems handle various stages of throttling. For testing software that does not allow for extended duration test sessions, then a script can be used to automatically repeat the test.

Performance testing can target either the CPU or GPU separately, or can simultaneously target both the CPU and GPU. Some example workload applications and programs are provided below:

• CPU and GPU for Apple Silicon and Intel: Cinebench.
  
  
• CPU and GPU for Apple Silicon and Intel: Geekbench.
  
  
• CPU for Apple Silicon and Intel: Endurance CPU Stress Test.

Back to top

Testing Variables

To achieve useful and accurate results, testing conditions need to be as close to identical between the CONTROL and SVALT tests as possible. This is accomplished by eliminating non-testing variables and setting up tests in a way that can be repeated as easily as possible with identical workloads and conditions. The following are the conditions and variables that SVALT controls for its tests:

Laptop Compatibility
Use a compatible laptop model for which the SVALT product has been optimized as described on the SVALT product page or as listed in the compatibility chart on the compare page. To better understand how Apple laptop’s built-in cooling system works, including how it works with SVALT products, see Laptops.

Product Setup
Proper laptop setup with the SVALT product is essential for viable testing as it has a significant impact on cooling performance. Please see the SVALT Product Setup page. Note that for testing, SVALT metal and heat sink components will absorb heat, so do not setup the laptop with the SVALT product until ready to begin the SVALT test session.

Workstation Setup
Proper workstation setup with the laptop and SVALT product is also essential for viable testing as it has a significant impact on cooling performance. Please see the SVALT Workstation Setup page. Note that only the workstation layouts labeled as “best” should be used for testing and any workstation setup labeled as “avoid” should not be included in testing.

Background Activities
There are a number of automated background activities that can consume system resources, generate significant amounts of heat and reduce performance. The following steps can be taken to help ensure that nothing other than the intended test workload is stressing the system, and to help ensure that test workloads are consistent for all tests (note that some of these settings will automatically reset, so check before each test session):

• Disconnect Ethernet, external hard drives, and any other peripheries not required for testing.
  
  
• Turn off Wi-Fi.
  
  
• Turn off Bluetooth if you are able to use wired keyboard and mouse.
  
  
• Turn off TimeMachine, DropBox and other automated backups.
  
  
• Open System Settings -> Focus -> select Do Not Disturb.
  
  
• Open System Settings -> Battery -> On power adapter -> select High Power.
  
  
• Open System Settings -> Displays -> deselect Automatic adjust brightness -> slide to max brightness.
  
  
• Open System Settings -> Lock Screen -> Start Screen Saver when inactive -> select Never.
  
  
• Open System Settings -> Lock Screen -> Turn display off on power adapter when inactive -> select Never.
  
  
• Open System Settings -> Privacy & Security -> Analytics & Improvements -> opt out of various automatic monitoring and reporting.
  
  
• To avoid Spotlight from influencing testing make sure to let the computer run idle for a few minutes after startup or restart. Note that brand new laptops or OS re-installs may requires hours or days to complete the initial Spotlight indexing.

Laptop Built-in Fan Control
SVALT supplemental cooling expands the laptop’s cooling capacity to point that the laptop will export cooling from the laptop’s built-in cooling system to SVALT’s supplemental cooling system. That can be beneficial for reducing laptop fan noise and for long term laptop component durability with reduced demands and stresses on the laptop’s built-in cooling system, but it makes isolating and measuring the influence of supplemental cooling more difficult. There are two options, either collect fan speed data and factor that into test results, or eliminate the fan speed variable by matching fan speeds for both the CONTROL and the SVALT test. In the CONTROL tests the laptop fans can run 25-50% faster RPMs compared to SVALT tests that benefit from supplemental cooling. Locking laptop fan RPMs at the CONTROL test’s higher fan speed normalizes this variable. In temperature testing with light workloads, the laptop fans can be left in system default/auto and monitored in case tests need to be repeated. In performance testing with heavy workloads, the laptop fans can be locked at maximum RPM right before the test session begins. Controlling fan speeds is considered an advanced setting and fans should never be allowed to spin at speeds lower than the system default speeds. There are free and paid applications that allow for the fans to be manually controlled.

• For Apple Silicon and Intel chip Macs: Paddle TG Pro
  
  
• For Apple Silicon and Intel chip Macs: Macs Fan Control

Laptop Starting Temperatures
Match internal component temperatures prior to starting tests. The easiest way to achieve this is to start the first test at normal working temperatures after conducting some light workloads. This can often be accomplished by setting up the laptop system for testing, opening test apps and arranging windows. Once everything is setup then take a screenshot of internal component temperatures for a reference to use during the next test. After completing the first test, the laptop may need to sit idle or power down for a period of time before matching the first test’s screenshot reference temperatures. This could take minutes to hours. Note that the SVALT product slated to be used for testing should not be used to cool the laptop if it will be used in the second test as aluminum components will absorb heat not associated with the testing workload. Also note that the laptop battery should be fully charged at least an hour before starting tests because charging the battery, or a recently charged battery, can add significant heat to the system.

Ambient Temperatures
Match ambient temperatures prior to starting tests and maintain through both the CONTROL and SVALT tests. Since most electrical components generate heat that can be transferred to the environment, the laptop and the SVALT product (see the Workstation Setup page), it is critical that external ambient temperature gauges, the laptop and the SVALT supplemental cooling device are all located so that external heating or cooling sources do influence temperatures. To maintain a consistent ambient temperature during testing, an environmental control system zoned for a single room or a manually controlled heating cooling system will likely be required. Note that roughly 75 Fahrenheit temperatures are typically used for SVALT testing. Also note that SVALT review products include an ambient temperature gauge.

Room and Furnishings
Testing on a non-conductive surface is critical to isolating supplemental cooling. Testing on a sheet of rigid insulation is ideal, otherwise testing on empty cardboard boxes can also work. The key is to avoid testing on a surface like a stone, concrete, glass or metal surface. You may also need to move furniture to clear the area around testing (see the Workstation Setup page).

Back to top

Testing Data

Processors change their speed and power at high rates and the changes can be dramatic from moment to moment. These changes have a profound impact on heat generation and temperatures. As a result it is not possible to gather any useful information by watching realtime sensor data or capturing a single moment of testing. The only way to gather meaningful test results is to collect the raw sensor data in log files that can be averaged from milliseconds into larger and more meaningful 1, 5, 10, 15, 30 and 60 minute averages.

Laptop Component Monitoring
To test the influence of SVALT supplemental cooling on laptop temperatures and performance, accurate laptop component temperature and power measurements need to be recorded and saved in a format that can be analyzed for comparison.

• For Apple Silicon and Intel chip Macs: Paddle TG Pro.
  
  
• For Apple Silicon chip Macs: Mx Power Gadget.
  
  
• For Intel chip Macs: Intel Performance Counter Monitor.

Testing Reference Data
While the above data sets will be used to calculate test results, it can be helpful to capture additional data for reference. For instance, it will be helpful to capture screen videos, screenshots and environmental photos for time and ambient temperature references. This can help you record testing start and stop times, as well as provide a general test reference for what was tested.

• Screen capture: Apple’s QuickTime Player can record a video of the screen.
  
  
• Environmental photos: A video, automated photo intervals, or snapshots at the start and other points of the test can be helpful to track testing setups and synchronize test data.
  
  
• Ambient temperature: Multiple thermistors can be used to measure temperatures around the laptop, and remote wired thermistors can be used to measure on surface temperatures.
  
  
• Infrared imaging: There’s a possibility to collect meaningful data if fixed in place with the ability to capture a larger data set, but IR images are more likely to be fun visual representations of general conditions than provide meaningful test data.
  
  
• Apple Activity Monitor provides real-time display and post-test reference through screen videos and shots of system activity (included with macOS) to help you identify non-tested workloads that could result in a test session failure.

Back to top

Testing Process

The following provides a sequential step-by-step guide for testing SVALT supplemental cooling. The CONTROL test can occur before or after the SVALT test, but is listed here as occurring after the SVALT test for an easier and much quicker post-test cool down. Either way, a sufficient amount of time is required to allow the laptop to cool down to pre-testing conditions before conducting an additional test session.

1. Establish a testing ambient temperature that can be maintained throughout both the SVALT and CONTROL tests.
   
   
2. Restrict laptop activities and maintain ambient temperature per the Variables section.
   
   
3. Open selected testing application and prepare application for test, but do not start test per the Types section.
   
   
4. Open laptop monitoring software, arrange sensor windows so that they can be captured in screen videos and screenshots per the Variables and Data sections.
   
   
5. Test setup:
   
   
   • 1st SVALT test only: Set laptop onto SVALT supplemental cooling device per the Variables section.
     
     
   • 2nd CONTROL test only: Match laptop temperatures with the 1st SVALT pre-test temperatures. If temperatures are higher, then wait at idle for temperatures to drop. If temperatures are lower, then apply light workload. See the Variables and Data sections.
     
     
6. Start all data recording per the Variables and Data sections.
   
   
7. Adjust laptop internal fan speeds per the Variables section.
   
   
8. Start testing workload per the Types section.
   
   
9. After testing workload is complete, then end workloads and quit applications per the Types section.
   
   
10. Return internal fan speeds and System Settings to your pre-test settings or defaults per the Variables section.
    
    
11. Save data logs per the Data section.
    
    
12. Test wrap up:
    
    
    • 1st SVALT test only: Cool off laptop to return to pre-test temperatures, and then repeat testing for the 2nd CONTROL test with the same ambient temperature and laptop testing conditions per the Variables section.
      
      
    • 2nd CONTROL test only: Sort, average, analyze and compare testing data.
      
      

Back to top