You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-23 - 23:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Thu Oct 23, 2025 12:44:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
387821050,2chrt3877-21users07:33:321
387821050,2chrt3877-21users07:33:321
1008621020,2chrt0-21swapper/007:53:090
270692900,2sleep0201rcuc/011:28:240
270692900,2sleep0201rcuc/011:28:240
182552880,2sleep118256-21needreboot09:38:271
182552880,2sleep118256-21needreboot09:38:271
127232730,3sleep012726-21unixbench-2d10:48:360
127232730,3sleep012726-21unixbench-2d10:48:360
189562630,3chrt0-21swapper/012:33:160
189562630,3chrt0-21swapper/012:33:160
2737399554,9cyclictest181rcu_preempt12:23:231
2737399554,9cyclictest181rcu_preempt12:23:231
2737399520,8cyclictest181rcu_preempt09:48:201
2737399520,8cyclictest181rcu_preempt09:48:201
147802510,2sleep00-21swapper/009:28:330
147802510,2sleep00-21swapper/009:28:330
2737399504,13cyclictest181rcu_preempt09:33:191
2737399504,13cyclictest181rcu_preempt09:33:191
2737399501,5cyclictest181rcu_preempt11:43:201
2737399501,5cyclictest181rcu_preempt11:43:201
2737399493,6cyclictest181rcu_preempt08:38:201
2737399462,9cyclictest181rcu_preempt09:43:181
2737399462,9cyclictest181rcu_preempt09:43:181
2737399441,5cyclictest181rcu_preempt10:13:211
2737399441,5cyclictest181rcu_preempt10:13:211
31992430,3sleep30-21swapper/308:58:293
48082420,5chrt0-21swapper/011:53:320
48082420,5chrt0-21swapper/011:53:320
2737299423,35cyclictest18579-21kworker/0:3+events_power_efficient12:38:210
2737299423,35cyclictest18579-21kworker/0:3+events_power_efficient12:38:210
291432410,3sleep00-21swapper/008:43:180
2737399411,5cyclictest181rcu_preempt11:18:201
2737399411,5cyclictest181rcu_preempt11:18:201
27372994133,6cyclictest0-21swapper/010:18:220
27372994133,6cyclictest0-21swapper/010:18:220
27372994133,5cyclictest0-21swapper/011:13:240
27372994133,5cyclictest0-21swapper/011:13:240
27372994115,18cyclictest201rcuc/009:13:190
142472400,3sleep30-21swapper/308:03:253
142472400,3sleep30-21swapper/308:03:253
27372993914,4cyclictest201rcuc/011:52:470
27372993914,4cyclictest201rcuc/011:52:470
2737299387,27cyclictest55150irq/47-eth007:53:200
2737299387,27cyclictest55150irq/47-eth007:53:200
2737299384,29cyclictest4622-21sshd07:38:190
2737299382,4cyclictest201rcuc/007:58:200
2737299382,4cyclictest201rcuc/007:58:200
27373993732,3cyclictest6252-21latency_hist07:43:071
2737399371,6cyclictest181rcu_preempt10:38:191
2737399371,6cyclictest181rcu_preempt10:38:191
27372993713,5cyclictest201rcuc/009:23:540
27372993713,5cyclictest201rcuc/009:23:540
286002360,2sleep10-21swapper/111:33:161
286002360,2sleep10-21swapper/111:33:161
27372993620,12cyclictest171ktimers/007:43:210
2737299361,6cyclictest201rcuc/008:03:220
2737299361,6cyclictest201rcuc/008:03:220
27372993611,22cyclictest201rcuc/009:00:240
27373993525,8cyclictest7213-21netstat07:43:281
2737399351,27cyclictest706-21sed07:28:161
2737299359,4cyclictest201rcuc/008:19:540
2737299359,4cyclictest201rcuc/008:19:540
27372993514,5cyclictest201rcuc/008:16:540
27372993514,5cyclictest201rcuc/008:16:540
2737299344,23cyclictest663-21runrttasks07:33:200
2737299344,23cyclictest663-21runrttasks07:33:200
2737299344,22cyclictest31045-21taskset08:48:200
27372993412,5cyclictest201rcuc/012:26:230
27372993412,5cyclictest201rcuc/012:26:230
27372993412,5cyclictest201rcuc/008:23:530
27372993412,5cyclictest201rcuc/008:23:530
27372993410,5cyclictest201rcuc/012:20:540
27372993410,5cyclictest201rcuc/012:20:540
237302340,2sleep10-21swapper/108:28:241
13932340,2sleep30-21swapper/308:53:303
13932340,2sleep30-21swapper/308:53:303
316552330,2sleep0201rcuc/010:13:310
316552330,2sleep0201rcuc/010:13:310
27373993328,3cyclictest27370-21cyclictest11:53:241
27373993328,3cyclictest27370-21cyclictest11:53:241
27373993327,4cyclictest0-21swapper/108:53:141
2737299336,24cyclictest55150irq/47-eth009:08:190
2737299336,24cyclictest55150irq/47-eth009:08:190
2737299332,25cyclictest15468-21sshd10:58:240
2737299332,25cyclictest15468-21sshd10:58:240
27372993315,5cyclictest201rcuc/009:18:540
27372993315,5cyclictest201rcuc/009:18:540
27373993224,6cyclictest11935-21kernelversion12:13:251
2737299323,23cyclictest663-21runrttasks11:23:230
2737299323,23cyclictest663-21runrttasks11:23:230
27372993225,5cyclictest407-21in:imuxsock10:44:160
27372993225,5cyclictest407-21in:imuxsock10:44:160
27372993224,6cyclictest32738-21fschecks_count11:43:220
27372993224,6cyclictest32738-21fschecks_count11:43:220
27372993218,7cyclictest201rcuc/010:03:200
27372993212,5cyclictest201rcuc/011:05:540
27372993212,5cyclictest201rcuc/011:05:540
27372993211,5cyclictest201rcuc/011:40:540
27372993211,5cyclictest201rcuc/011:40:540
27372993210,5cyclictest201rcuc/009:41:540
27372993210,5cyclictest201rcuc/009:41:540
27373993126,4cyclictest32321-21grep11:43:081
27373993126,4cyclictest32321-21grep11:43:081
27373993124,5cyclictest12736-21unixbench-2d10:48:361
27373993124,5cyclictest12736-21unixbench-2d10:48:361
27372993124,5cyclictest25198-21latency_hist08:33:100
27372993123,6cyclictest7934-21fschecks_time12:03:240
27372993123,6cyclictest7934-21fschecks_time12:03:240
27372993023,5cyclictest21706-21head09:48:240
27372993023,5cyclictest21706-21head09:48:240
27372993022,6cyclictest7984-21cut10:38:210
27372993022,6cyclictest7984-21cut10:38:210
27372993022,5cyclictest28264-21threads08:38:310
27372993012,16cyclictest201rcuc/009:07:540
27372993012,16cyclictest201rcuc/009:07:540
27374992918,7cyclictest11079-21chrt07:53:322
27374992918,7cyclictest11079-21chrt07:53:322
27373992924,3cyclictest1055-21gltestperf07:28:251
27373992924,3cyclictest1055-21gltestperf07:28:251
27373992924,3cyclictest0-21swapper/108:53:311
27373992924,3cyclictest0-21swapper/108:53:311
27373992922,5cyclictest29112-21fschecks_time10:08:221
27373992922,5cyclictest29112-21fschecks_time10:08:221
27373992922,5cyclictest17813-21if_wlan011:03:251
27373992922,5cyclictest17813-21if_wlan011:03:251
2737399291,6cyclictest13611-21192.168.115.40-manager08:00:501
2737399291,6cyclictest13611-21192.168.115.40-manager08:00:501
27372992922,5cyclictest25366-21fschecks_time09:58:220
27372992922,5cyclictest25366-21fschecks_time09:58:220
27372992911,16cyclictest201rcuc/012:10:540
27372992911,16cyclictest201rcuc/012:10:540
40232280,1sleep04026-21sort10:28:180
27373992823,2cyclictest0-21swapper/111:33:291
27373992823,2cyclictest0-21swapper/111:33:291
2737399282,21cyclictest19708-21taskset08:18:211
2737399282,21cyclictest19708-21taskset08:18:211
2737299289,3cyclictest201rcuc/009:45:540
2737299289,3cyclictest201rcuc/009:45:540
27372992822,4cyclictest1913-21sshd08:54:020
27372992822,4cyclictest1913-21sshd08:54:020
27372992818,2cyclictest181rcu_preempt11:18:260
27372992818,2cyclictest181rcu_preempt11:18:260
27372992813,4cyclictest91-1pr/ttyS008:35:240
27372992813,4cyclictest91-1pr/ttyS008:35:240
27375992716,8cyclictest8894-21chrt07:48:263
27373992720,5cyclictest383-21cron10:58:091
27373992720,5cyclictest17615-21kernelversion12:28:261
27373992720,5cyclictest17615-21kernelversion12:28:261
27373992720,4cyclictest0-21swapper/110:03:241
27373992719,5cyclictest394-21snmpd09:23:181
27373992719,5cyclictest394-21snmpd09:23:181
27373992716,8cyclictest10522-21kernelversion09:18:271
27373992716,8cyclictest10522-21kernelversion09:18:271
2737299279,8cyclictest16-21ksoftirqd/012:16:460
2737299278,4cyclictest201rcuc/011:38:080
2737299278,4cyclictest201rcuc/011:38:080
27372992721,4cyclictest146-21systemd-journal12:00:230
27372992721,4cyclictest146-21systemd-journal12:00:230
27372992717,2cyclictest181rcu_preempt12:33:290
27372992717,2cyclictest181rcu_preempt12:33:290
2737399267,3cyclictest181rcu_preempt10:59:541
2737399267,3cyclictest181rcu_preempt10:59:541
27373992622,2cyclictest0-21swapper/112:33:311
27373992622,2cyclictest0-21swapper/112:33:311
27373992621,3cyclictest0-21swapper/110:33:341
27373992621,3cyclictest0-21swapper/110:33:341
27373992621,3cyclictest0-21swapper/110:33:341
27373992621,3cyclictest0-21swapper/109:03:071
27373992619,5cyclictest30801-21latency_hist08:48:081
27373992619,5cyclictest2738-21ntp_kernel_pll_11:48:291
27373992619,5cyclictest2738-21ntp_kernel_pll_11:48:291
27373992619,5cyclictest13991-21needreboot12:18:291
27373992619,5cyclictest13991-21needreboot12:18:291
2737299266,4cyclictest201rcuc/007:26:500
27372992621,3cyclictest16077-21fschecks_count08:08:220
27372992619,5cyclictest14801-21sshd10:54:230
27372992614,2cyclictest181rcu_preempt10:08:210
27372992614,2cyclictest181rcu_preempt10:08:210
27372992610,3cyclictest181rcu_preempt11:08:240
27372992610,3cyclictest181rcu_preempt11:08:240
60622250,2chrt0-21swapper/209:08:092
60622250,2chrt0-21swapper/209:08:092
27373992521,3cyclictest27370-21cyclictest12:08:281
27373992521,3cyclictest27370-21cyclictest12:08:281
27373992520,3cyclictest0-21swapper/109:08:321
27373992520,3cyclictest0-21swapper/109:08:321
27373992519,3cyclictest0-21swapper/111:08:231
27373992519,3cyclictest0-21swapper/111:08:231
27373992519,3cyclictest0-21swapper/109:28:231
27373992519,3cyclictest0-21swapper/109:28:231
27373992516,7cyclictest10445-21fschecks_count07:53:241
27373992516,7cyclictest10445-21fschecks_count07:53:241
2737299255,4cyclictest181rcu_preempt09:36:070
2737299255,4cyclictest181rcu_preempt09:36:070
27372992520,3cyclictest0-21swapper/010:33:240
27372992520,3cyclictest0-21swapper/010:33:240
27372992520,3cyclictest0-21swapper/010:33:240
27372992515,3cyclictest181rcu_preempt10:23:250
27372992515,3cyclictest181rcu_preempt10:23:250
27372992513,4cyclictest181rcu_preempt09:58:150
27372992513,4cyclictest181rcu_preempt09:58:150
2737499243,11cyclictest0-21swapper/209:18:232
2737499243,11cyclictest0-21swapper/209:18:232
27373992419,2cyclictest0-21swapper/108:33:321
27373992419,2cyclictest0-21swapper/108:33:321
27373992418,4cyclictest830-21missed_timers10:18:291
27373992418,4cyclictest830-21missed_timers10:18:291
27373992418,3cyclictest0-21swapper/110:28:221
27373992418,3cyclictest0-21swapper/108:08:561
27373992417,4cyclictest4739-21gltestperf09:03:221
27373992417,4cyclictest4739-21gltestperf09:03:221
27373992417,4cyclictest0-21swapper/108:13:241
27373992417,4cyclictest0-21swapper/108:13:241
27373992415,7cyclictest15696-21cron08:08:091
27373992415,7cyclictest15696-21cron08:08:091
2737299241,13cyclictest0-21swapper/007:28:220
2737299241,13cyclictest0-21swapper/007:28:220
27373992319,2cyclictest0-21swapper/111:13:371
27373992319,2cyclictest0-21swapper/111:13:371
27373992318,3cyclictest0-21swapper/112:38:321
27373992318,3cyclictest0-21swapper/112:38:321
27373992317,3cyclictest0-21swapper/109:53:371
27373992317,3cyclictest0-21swapper/109:53:371
27373992317,3cyclictest0-21swapper/107:48:341
2737399231,6cyclictest2635-21irqstats10:23:251
2737399231,6cyclictest2635-21irqstats10:23:251
2737399231,6cyclictest223552ntp_states08:23:281
2737399231,6cyclictest223552ntp_states08:23:281
27373992315,6cyclictest25572-21head11:23:301
27373992315,6cyclictest25572-21head11:23:301
27373992216,3cyclictest0-21swapper/112:03:241
27373992216,3cyclictest0-21swapper/112:03:241
27373992215,5cyclictest8773-21missed_timers09:13:261
27373992215,4cyclictest0-21swapper/110:43:241
27373992215,4cyclictest0-21swapper/110:43:241
215422220,2sleep20-21swapper/208:23:142
215422220,2sleep20-21swapper/208:23:142
27375992012,6cyclictest13566-21chrt12:18:243
27375992012,6cyclictest13566-21chrt12:18:243
2737599201,13cyclictest15946-21taskset08:08:203
2737499203,9cyclictest0-21swapper/211:23:232
2737499203,9cyclictest0-21swapper/211:23:232
2737499203,9cyclictest0-21swapper/210:23:212
2737499203,9cyclictest0-21swapper/210:23:212
2737499203,9cyclictest0-21swapper/207:28:142
2737499203,8cyclictest0-21swapper/212:03:222
2737499203,8cyclictest0-21swapper/212:03:222
2737499202,9cyclictest0-21swapper/209:43:202
2737499202,9cyclictest0-21swapper/209:43:202
2737499201,12cyclictest26076-21chrt08:33:272
2737499201,12cyclictest26076-21chrt08:33:272
27373992015,3cyclictest60-21kcompactd011:58:241
27373992015,3cyclictest60-21kcompactd011:58:241
2737399201,3cyclictest25126-21apt-get09:58:191
2737399201,3cyclictest25126-21apt-get09:58:191
2737599193,9cyclictest0-21swapper/311:33:213
2737599193,9cyclictest0-21swapper/311:33:213
2737599193,9cyclictest0-21swapper/310:58:173
2737599192,12cyclictest23007-21chrt11:18:223
2737599192,12cyclictest23007-21chrt11:18:223
2737499193,9cyclictest0-21swapper/209:38:202
2737499193,9cyclictest0-21swapper/209:38:202
2737499193,9cyclictest0-21swapper/209:28:212
2737499193,9cyclictest0-21swapper/209:28:212
2737499193,9cyclictest0-21swapper/208:43:202
2737499193,8cyclictest0-21swapper/212:33:212
2737499193,8cyclictest0-21swapper/212:33:212
2737499193,8cyclictest0-21swapper/208:13:212
2737499193,8cyclictest0-21swapper/208:13:212
2737499192,9cyclictest0-21swapper/212:18:222
2737499192,9cyclictest0-21swapper/212:18:222
2737499192,9cyclictest0-21swapper/209:33:202
2737499192,9cyclictest0-21swapper/209:33:202
2737499191,12cyclictest1347-21chrt08:53:292
2737499191,12cyclictest1347-21chrt08:53:292
131982190,3sleep20-21swapper/209:23:322
131982190,3sleep20-21swapper/209:23:322
2737599182,12cyclictest31045-21chrt08:48:203
27375991813,3cyclictest199452sleep309:43:243
27375991813,3cyclictest199452sleep309:43:243
2737599181,13cyclictest2558-21chrt07:33:143
2737599181,13cyclictest2558-21chrt07:33:143
2737499183,9cyclictest0-21swapper/210:58:142
2737499183,9cyclictest0-21swapper/210:18:242
2737499183,9cyclictest0-21swapper/210:18:242
2737499183,9cyclictest0-21swapper/209:58:202
2737499183,9cyclictest0-21swapper/209:58:202
2737499183,9cyclictest0-21swapper/208:53:162
2737499183,8cyclictest0-21swapper/211:58:202
2737499183,8cyclictest0-21swapper/211:58:202
2737499183,8cyclictest0-21swapper/211:53:212
2737499183,8cyclictest0-21swapper/211:53:212
2737499183,8cyclictest0-21swapper/211:18:222
2737499183,8cyclictest0-21swapper/211:18:222
2737499183,8cyclictest0-21swapper/207:33:202
2737499183,8cyclictest0-21swapper/207:33:202
2737499181,9cyclictest0-21swapper/211:43:202
2737499181,9cyclictest0-21swapper/211:43:202
2737499181,12cyclictest16604-21chrt10:58:352
2737499181,12cyclictest16604-21chrt10:58:352
2737499181,11cyclictest14165-21chrt08:03:242
2737499181,11cyclictest14165-21chrt08:03:242
2737599173,8cyclictest0-21swapper/308:18:213
2737599173,8cyclictest0-21swapper/308:18:213
2737599173,10cyclictest0-21swapper/307:33:203
2737599173,10cyclictest0-21swapper/307:33:203
27375991711,4cyclictest0-21swapper/307:28:123
2737499173,8cyclictest0-21swapper/212:08:222
2737499173,8cyclictest0-21swapper/212:08:222
2737499173,8cyclictest0-21swapper/211:28:222
2737499173,8cyclictest0-21swapper/211:28:222
2737499171,11cyclictest6142-21chrt10:33:232
2737499171,11cyclictest6142-21chrt10:33:232
2737499171,11cyclictest6142-21chrt10:33:232
2737599163,9cyclictest0-21swapper/312:08:223
2737599163,9cyclictest0-21swapper/312:08:223
2737599163,9cyclictest0-21swapper/311:08:223
2737599163,9cyclictest0-21swapper/311:08:223
2737599163,9cyclictest0-21swapper/310:33:213
2737599163,9cyclictest0-21swapper/310:33:213
2737599163,9cyclictest0-21swapper/310:33:213
2737599163,9cyclictest0-21swapper/310:28:203
2737599163,9cyclictest0-21swapper/309:48:213
2737599163,9cyclictest0-21swapper/309:48:213
2737599163,9cyclictest0-21swapper/308:38:213
2737599161,10cyclictest28921-21chrt10:08:193
2737599161,10cyclictest28921-21chrt10:08:193
27375991610,4cyclictest125442sleep310:48:353
27375991610,4cyclictest125442sleep310:48:353
2737499163,9cyclictest0-21swapper/212:23:232
2737499163,9cyclictest0-21swapper/212:23:232
2737499163,9cyclictest0-21swapper/211:48:212
2737499163,9cyclictest0-21swapper/211:48:212
2737499163,9cyclictest0-21swapper/209:08:212
2737499163,9cyclictest0-21swapper/209:08:212
2737499163,9cyclictest0-21swapper/208:23:202
2737499163,9cyclictest0-21swapper/208:23:202
2737499163,9cyclictest0-21swapper/207:58:222
2737499163,9cyclictest0-21swapper/207:58:222
2737499163,9cyclictest0-21swapper/207:28:222
2737499163,9cyclictest0-21swapper/207:28:222
2737499161,9cyclictest8408-21taskset07:48:212
2737499161,11cyclictest30805-21chrt10:13:212
2737499161,11cyclictest30805-21chrt10:13:212
2737499161,11cyclictest27035-21chrt10:03:202
2737499161,11cyclictest20889-21chrt12:38:212
2737499161,11cyclictest20889-21chrt12:38:212
2737499161,11cyclictest0-21swapper/211:08:152
2737499161,11cyclictest0-21swapper/211:08:152
2737499161,10cyclictest4044-21chrt10:28:202
2737499161,10cyclictest11751-21taskset10:48:262
2737499161,10cyclictest11751-21taskset10:48:262
2737599153,9cyclictest0-21swapper/311:43:203
2737599153,9cyclictest0-21swapper/311:43:203
2737599153,9cyclictest0-21swapper/310:38:193
2737599153,9cyclictest0-21swapper/310:38:193
2737599153,8cyclictest0-21swapper/310:13:213
2737599153,8cyclictest0-21swapper/310:13:213
2737599152,9cyclictest0-21swapper/308:33:203
2737599152,9cyclictest0-21swapper/308:33:203
2737599152,11cyclictest0-21swapper/309:38:313
2737599152,11cyclictest0-21swapper/309:38:313
2737599152,10cyclictest23522-21chrt09:53:243
2737599152,10cyclictest23522-21chrt09:53:243
2737599151,12cyclictest0-21swapper/311:28:233
2737599151,12cyclictest0-21swapper/311:28:233
2737599151,11cyclictest25387-21chrt09:58:233
2737599151,11cyclictest25387-21chrt09:58:233
2737599151,11cyclictest19934-21chrt12:33:313
2737599151,11cyclictest19934-21chrt12:33:313
2737599151,11cyclictest0-21swapper/307:38:213
2737599151,10cyclictest21616-21chrt08:23:193
2737599151,10cyclictest21616-21chrt08:23:193
2737499154,4cyclictest0-21swapper/208:38:212
2737499153,9cyclictest0-21swapper/210:43:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional