You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 04:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Jul 08, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2879029582,9sleep00-21swapper/019:08:080
2817229573,16sleep30-21swapper/319:05:243
2872829367,19sleep10-21swapper/119:07:191
2875728975,9sleep20-21swapper/219:07:402
115342680,1sleep20-21swapper/221:41:232
2902799280,1cyclictest0-21swapper/300:36:123
2902799250,24cyclictest0-21swapper/300:33:203
2901999250,23cyclictest0-21swapper/100:30:131
2902799240,3cyclictest0-21swapper/322:27:333
2902399240,24cyclictest0-21swapper/223:05:092
2902399240,24cyclictest0-21swapper/222:09:362
2902399230,23cyclictest0-21swapper/222:43:322
2902399230,23cyclictest0-21swapper/222:43:322
2902799221,1cyclictest0-21swapper/322:57:553
2901999220,3cyclictest0-21swapper/123:44:341
2901999220,20cyclictest0-21swapper/121:54:211
2902799211,5cyclictest0-21swapper/321:36:293
2901999210,1cyclictest0-21swapper/100:03:371
2901799210,2cyclictest0-21swapper/023:12:100
2902799202,15cyclictest0-21swapper/300:29:103
2902799200,17cyclictest0-21swapper/300:05:523
2902399200,16cyclictest0-21swapper/200:10:452
217432200,0sleep00-21swapper/023:24:030
2902799190,3cyclictest0-21swapper/323:12:233
29023991910,6cyclictest21835-21diskmemload21:29:502
2901999194,15cyclictest28163-21kworker/1:200:21:591
2901999191,18cyclictest0-21swapper/122:31:451
2901799190,19cyclictest0-21swapper/021:41:130
2902799181,1cyclictest0-21swapper/323:28:273
2902799180,1cyclictest0-21swapper/321:30:073
2902399184,14cyclictest32-21ksoftirqd/223:15:062
29023991812,2cyclictest311rcuc/221:30:092
29023991810,7cyclictest0-21swapper/223:50:132
29023991810,2cyclictest32-21ksoftirqd/200:08:112
2901999183,10cyclictest1142-21runrttasks00:28:521
29027991716,1cyclictest0-21swapper/300:11:253
29027991715,1cyclictest15922-21sh00:17:583
2902799170,1cyclictest0-21swapper/322:09:323
2902799170,16cyclictest0-21swapper/300:03:373
2902399170,17cyclictest0-21swapper/200:21:152
2902399170,16cyclictest0-21swapper/200:04:442
2902399170,15cyclictest0-21swapper/223:35:132
2901999173,1cyclictest0-21swapper/122:51:401
29019991713,3cyclictest0-21swapper/122:27:091
2901999170,16cyclictest0-21swapper/100:36:091
2902799162,1cyclictest0-21swapper/321:44:583
2902799161,9cyclictest17176-21basename00:00:003
2902799160,1cyclictest0-21swapper/322:23:373
2902799160,0cyclictest0-21swapper/323:44:343
2902399166,10cyclictest0-21swapper/221:10:152
2902399165,10cyclictest0-21swapper/221:35:122
2902399163,10cyclictest32-21ksoftirqd/221:04:182
29023991614,1cyclictest25891-21apt-get20:10:142
29023991613,2cyclictest0-21swapper/222:54:042
29023991610,5cyclictest21835-21diskmemload21:56:152
2902399160,7cyclictest0-21swapper/223:30:102
2902399160,4cyclictest0-21swapper/222:48:132
2902399160,4cyclictest0-21swapper/222:48:132
2901999163,13cyclictest21835-21diskmemload23:04:451
2901999160,15cyclictest0-21swapper/121:31:541
2901799164,6cyclictest20190-21munin-run00:40:000
2901799161,8cyclictest15793-21munin-run23:40:000
29017991615,1cyclictest0-21swapper/022:42:340
29017991615,1cyclictest0-21swapper/022:42:330
29027991514,1cyclictest0-21swapper/300:21:593
2902799151,1cyclictest0-21swapper/323:18:323
2902799151,0cyclictest0-21swapper/323:06:293
2902799150,2cyclictest0-21swapper/321:26:183
2902799150,1cyclictest0-21swapper/321:54:213
2902799150,11cyclictest0-21swapper/320:25:113
2902399158,6cyclictest32-21ksoftirqd/200:35:502
2902399158,6cyclictest0-21swapper/221:47:282
2902399157,8cyclictest0-21swapper/223:42:192
2902399154,5cyclictest225-21systemd-journal20:03:192
2902399150,15cyclictest0-21swapper/222:35:282
2901999152,12cyclictest21835-21diskmemload23:17:461
2901999151,13cyclictest10972-21ssh22:20:131
2901999150,5cyclictest0-21swapper/122:05:111
2901799157,6cyclictest13-21ksoftirqd/023:29:090
2901799153,2cyclictest0-21swapper/022:00:130
29017991511,2cyclictest24783-21tail21:30:170
2901799150,1cyclictest0-21swapper/023:45:410
2901799150,1cyclictest0-21swapper/021:47:570
2902799144,5cyclictest0-21swapper/322:18:243
2902799141,1cyclictest0-21swapper/321:21:413
2902799140,3cyclictest0-21swapper/322:34:053
2902799140,1cyclictest0-21swapper/321:56:413
2902399144,10cyclictest0-21swapper/200:16:322
2902399143,5cyclictest29291-21sensors00:25:252
2902399143,4cyclictest21835-21diskmemload22:20:442
2902399142,5cyclictest0-21swapper/222:57:562
29023991411,2cyclictest22954-21turbostat23:25:002
29023991411,2cyclictest12793-21grep20:50:142
29023991410,3cyclictest32-21ksoftirqd/222:10:392
29023991410,3cyclictest21348-21grep22:25:272
29023991410,2cyclictest6549-21cat21:20:002
29019991413,1cyclictest0-21swapper/123:14:101
2901999140,3cyclictest27194-21grep20:10:231
2901999140,1cyclictest141rcu_preempt22:39:381
2901999140,1cyclictest0-21swapper/122:42:051
2901999140,1cyclictest0-21swapper/122:42:051
2901999140,14cyclictest0-21swapper/122:56:411
29017991413,1cyclictest0-21swapper/022:32:090
29017991410,3cyclictest711-21sh21:54:560
2901799140,8cyclictest18893-21basename22:25:000
2902799139,1cyclictest12456-21aten2_r7power_e22:40:133
2902799139,1cyclictest12456-21aten2_r7power_e22:40:133
2902799135,3cyclictest0-21swapper/323:01:373
2902799134,7cyclictest9879-21grep20:45:003
2902799134,5cyclictest26096-21memory23:45:193
2902799133,4cyclictest0-21swapper/321:45:123
2902799133,3cyclictest0-21swapper/320:15:153
29027991312,1cyclictest0-21swapper/323:32:583
2902799131,1cyclictest0-21swapper/321:10:293
2902799131,12cyclictest4088-21ssh21:18:133
29027991310,2cyclictest16937-21sed23:20:253
2902799130,13cyclictest0-21swapper/322:36:383
2902399134,1cyclictest0-21swapper/221:51:432
2902399131,11cyclictest4017-21cut00:30:142
29023991311,1cyclictest311rcuc/222:31:002
29023991310,2cyclictest30939-21grep23:10:142
29023991310,1cyclictest0-21swapper/220:55:132
2902399130,13cyclictest0-21swapper/219:33:282
2902399130,12cyclictest10556-21tr23:55:202
2901999138,4cyclictest3713-21cron19:25:001
2901999137,6cyclictest38150irq/131-snd_hda23:53:061
2901999137,5cyclictest20703-21sh23:23:151
2901999131,8cyclictest8057-21/usr/sbin/munin21:20:191
2901999131,11cyclictest25223-21tr23:45:141
29019991310,2cyclictest1757-21cron00:10:001
2901999130,8cyclictest0-21swapper/121:15:261
2901999130,1cyclictest141rcu_preempt22:45:591
2901999130,1cyclictest141rcu_preempt22:45:591
2901999130,12cyclictest0-21swapper/123:32:581
2901799139,3cyclictest32097-21grep23:30:120
29017991312,1cyclictest0-21swapper/023:08:570
29017991310,2cyclictest27279-21munin-run22:30:000
29017991310,2cyclictest15026-21cron21:25:000
29017991310,2cyclictest13437-21sh00:16:020
2902799129,2cyclictest6561-21ls20:35:223
2902799129,2cyclictest32348-21sed20:20:263
2902799129,2cyclictest32311-21cut19:15:193
2902799129,2cyclictest28914-21cron22:50:003
2902799129,2cyclictest28914-21cron22:50:003
2902799129,2cyclictest18106-21tail21:00:163
2902799129,2cyclictest1625-21basename19:20:133
2902799129,2cyclictest14965-21ssh23:39:233
2902799129,2cyclictest12819-21cat19:40:253
2902799122,1cyclictest0-21swapper/319:30:133
29027991210,1cyclictest21337-21grep21:05:253
2902799120,1cyclictest0-21swapper/322:12:323
2902799120,10cyclictest0-21swapper/322:03:173
2902399129,2cyclictest6972-21grep20:35:262
2902399129,2cyclictest3599-21ssh22:15:202
2902399129,2cyclictest23565-21grep23:25:152
2902399129,2cyclictest21823-21wc23:05:002
2902399129,2cyclictest21348-21grep21:05:252
2902399129,2cyclictest10918-21sed20:45:182
2902399128,3cyclictest21835-21diskmemload22:03:052
2902399127,2cyclictest21835-21diskmemload23:45:402
2901999129,2cyclictest4158-21grep00:10:271
2901999129,2cyclictest17831-21seq19:50:251
2901999129,2cyclictest15217-21tr20:55:151
2901999129,2cyclictest1474-21cut21:55:141
2901999128,3cyclictest28586-21awk22:10:261
2901999127,4cyclictest5707-21ssh21:38:231
2901999121,11cyclictest0-21swapper/121:45:131
29019991210,1cyclictest11905-21grep00:15:191
2901999120,1cyclictest0-21swapper/122:18:491
2901999120,11cyclictest9383-21grep23:55:131
2901999120,11cyclictest18090-21sed21:00:161
2901799129,2cyclictest6896-21cut23:15:160
2901799129,2cyclictest4734-21head00:30:160
2901799129,2cyclictest3253-21tail21:55:270
2901799129,2cyclictest319172sleep000:08:150
2901799129,2cyclictest26649-21grep22:10:130
2901799129,2cyclictest25481-21wc20:10:000
2901799129,2cyclictest24336-21wc23:45:010
2901799129,2cyclictest23440-21cron21:30:000
2901799129,2cyclictest21813-21cron21:10:000
2901799129,2cyclictest19667-21grep00:20:140
2901799129,2cyclictest14549-21grep19:45:180
2901799129,2cyclictest10890-21sed20:45:180
2901799129,2cyclictest10456-21lockfile-create22:20:000
2901799128,3cyclictest29051-21sed20:15:180
2901799128,3cyclictest22589-21sed20:00:250
2901799122,5cyclictest9866-21cron20:45:000
29017991211,1cyclictest13-21ksoftirqd/000:02:490
29017991211,1cyclictest0-21swapper/019:55:140
2902799119,2cyclictest5455-21cat19:25:253
2902799119,2cyclictest5455-21cat19:25:253
2902799119,1cyclictest715-21ssh23:50:073
2902799119,1cyclictest3615-21grep20:30:173
2902799119,1cyclictest29584-21expr19:10:133
2902799119,1cyclictest16593-21cat19:50:143
2902799119,1cyclictest10889-21grep20:45:183
2902799118,2cyclictest29893-21expr22:50:163
2902799118,2cyclictest15032-21basename20:55:153
2902799111,2cyclictest0-21swapper/320:00:123
2902399118,2cyclictest5246-21wc19:25:222
2902399118,2cyclictest5246-21wc19:25:222
2902399118,2cyclictest28975-21grep20:15:182
2902399118,2cyclictest20206-21grep19:55:272
2902399118,2cyclictest1160-21chrt19:18:072
2902399117,3cyclictest14057-21tr19:45:162
2902399116,4cyclictest32-21ksoftirqd/219:20:132
2902399115,5cyclictest30032-21gltestperf19:10:152
2902399114,5cyclictest32-21ksoftirqd/220:25:372
2902399114,5cyclictest32-21ksoftirqd/220:07:062
2902399113,8cyclictest0-21swapper/220:40:132
2902399112,2cyclictest32-21ksoftirqd/221:20:302
2902399111,3cyclictest32-21ksoftirqd/219:40:592
2901999118,2cyclictest8073-21wc20:40:141
2901999118,2cyclictest7373-21taskset20:38:521
2901999118,2cyclictest7175-21basename19:30:171
2901999118,2cyclictest628-21cut20:25:111
2901999118,2cyclictest4737-21expr19:25:171
2901999118,2cyclictest4737-21expr19:25:171
2901999118,2cyclictest3416-21cat20:30:161
2901999118,2cyclictest32571-21grep19:15:221
2901999118,2cyclictest27127-21ssh23:07:411
2901999118,2cyclictest25209-21chrt20:07:381
2901999118,2cyclictest23932-21grep21:10:221
2901999118,2cyclictest23259-21sh23:25:041
2901999118,2cyclictest21002-21grep20:00:121
2901999118,2cyclictest20845-21ntp_kernel_pll_21:05:191
2901999118,2cyclictest17055-21grep21:25:241
2901999118,2cyclictest15546-21chrt19:45:501
2901999118,2cyclictest15250-21sh22:03:071
2901999118,2cyclictest13410-21taskset20:50:181
2901999118,2cyclictest11061-21grep20:45:211
2901999117,3cyclictest7669-21ssh23:35:061
2901999114,2cyclictest11044-21cron19:40:001
29019991110,1cyclictest0-21swapper/121:40:041
2901799118,2cyclictest887-21ssh21:15:420
2901799118,2cyclictest5331-21ssh22:55:100
2901799118,2cyclictest5239-21systemctl22:35:220
2901799118,2cyclictest4091-21sed20:30:210
2901799118,2cyclictest30548-21sh00:26:070
2901799118,2cyclictest30474-21grep19:10:210
2901799118,2cyclictest30281-21grep22:50:180
2901799118,2cyclictest2509-21ssh21:35:530
2901799118,2cyclictest2207-21ssh00:10:090
2901799118,2cyclictest21888-21grep22:45:190
2901799118,2cyclictest21888-21grep22:45:190
2901799118,2cyclictest18455-21grep22:05:120
2901799118,2cyclictest18071-21chrt19:52:460
2901799118,2cyclictest11486-21grep19:40:140
2901799110,10cyclictest3184-21sh23:50:400
2901799110,10cyclictest22888-21tr21:10:150
101472111,5sleep013-21ksoftirqd/023:55:180
2902799108,1cyclictest18546-21basename19:55:143
2902799108,1cyclictest13984-21taskset19:45:153
2902799100,10cyclictest27804-21taskset20:12:423
2902399107,2cyclictest8825-21apt-config19:35:012
2902399106,3cyclictest32-21ksoftirqd/220:20:122
2902399103,4cyclictest5100-21munin-run20:35:012
2901999108,2cyclictest29144-21cat20:15:191
2901799108,1cyclictest8139-21taskset19:30:550
2901799108,1cyclictest53302sleep020:35:090
2901799108,1cyclictest3974-21cat19:25:130
2901799108,1cyclictest3974-21cat19:25:120
2901799108,1cyclictest26699-21idleruntime20:10:180
2901799108,1cyclictest15544-21chrt20:55:240
2901799108,1cyclictest143952sleep020:51:320
2901799108,1cyclictest13726-21ssh23:00:100
2901799107,2cyclictest2400-21hddtemp_smartct19:20:170
2901799107,2cyclictest19102-21taskset21:00:450
2901799107,2cyclictest1776-21open_files20:25:190
2901799100,9cyclictest31822-21ntp_states20:20:220
290279998,1cyclictest13526-21chrt20:50:193
290279992,4cyclictest8460-21cron19:35:043
290279990,9cyclictest24287-21ls20:05:193
290239997,1cyclictest17068-21kernelversion19:50:192
290199998,1cyclictest30952-21tr20:20:141
290199997,1cyclictest30849-21date19:10:251
290199996,2cyclictest19612-21cut19:55:201
290199996,2cyclictest12063-21grep19:40:161
290179997,1cyclictest8877-21cp19:35:010
290179996,2cyclictest31673-21tr19:15:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional