You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 19:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu Mar 26, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1377899260,25cyclictest0-21swapper/210:47:422
1377399262,2cyclictest313-21systemd-journal09:55:011
1378099250,24cyclictest7540-21diskmemload12:18:273
13778992523,1cyclictest0-21swapper/211:17:332
1377899250,0cyclictest0-21swapper/209:31:542
1376999253,0cyclictest62550irq/127-eno110:33:200
13769992521,3cyclictest9365-21ssh09:12:340
1376999250,25cyclictest0-21swapper/012:30:500
1376999250,24cyclictest0-21swapper/011:11:290
13780992423,1cyclictest22617-21ssh12:07:303
1378099240,23cyclictest0-21swapper/310:44:083
13778992423,0cyclictest26979-21ssh12:11:272
13778992422,1cyclictest7540-21diskmemload11:13:072
13778992422,1cyclictest0-21swapper/212:18:512
13778992421,0cyclictest0-21swapper/211:50:542
13778992420,4cyclictest0-21swapper/210:52:202
1377899240,23cyclictest17075-21sh10:57:302
1377899240,1cyclictest151rcu_preempt09:20:562
13773992423,1cyclictest21261-21sh09:56:441
13773992422,1cyclictest126522sleep111:24:321
13773992421,2cyclictest15536-21cut08:20:011
1377399240,24cyclictest7540-21diskmemload10:35:541
1377399240,24cyclictest0-21swapper/111:46:191
1377399240,24cyclictest0-21swapper/110:30:501
1377399240,24cyclictest0-21swapper/110:10:491
1377399240,23cyclictest22480-21ssh11:34:131
1377399240,23cyclictest19381-21ssh10:27:531
1377399240,23cyclictest10251-21sh11:53:581
1377399240,0cyclictest0-21swapper/109:17:401
1377399240,0cyclictest0-21swapper/109:13:101
13769992424,0cyclictest0-21swapper/009:44:500
13769992423,1cyclictest27346-21ssh12:12:110
13769992422,1cyclictest0-21swapper/009:50:390
13769992421,2cyclictest1762-21ssh11:46:040
1376999240,24cyclictest0-21swapper/010:41:400
1376999240,24cyclictest0-21swapper/010:36:400
1376999240,1cyclictest12187-21ssh10:19:340
1376999240,0cyclictest7540-21diskmemload11:42:190
1376999240,0cyclictest6038-21ssh10:46:150
13780992323,0cyclictest7540-21diskmemload12:00:163
13780992323,0cyclictest0-21swapper/312:39:493
13780992323,0cyclictest0-21swapper/312:26:403
13780992323,0cyclictest0-21swapper/311:52:373
13780992323,0cyclictest0-21swapper/311:49:543
13780992322,1cyclictest29147-21diskmemload09:32:233
13780992322,1cyclictest28816-21sh10:36:573
13780992321,1cyclictest0-21swapper/311:58:363
13780992320,2cyclictest29525-21rm11:08:413
13780992320,2cyclictest19073-21rm11:31:283
13780992320,2cyclictest18758-21sh09:21:353
1378099230,2cyclictest1956-21ssh10:09:003
1378099230,23cyclictest7540-21diskmemload11:03:283
1378099230,23cyclictest31071-21ssh11:10:503
1378099230,23cyclictest0-21swapper/311:22:273
1378099230,23cyclictest0-21swapper/310:12:543
1378099230,23cyclictest0-21swapper/309:10:443
1378099230,22cyclictest29931-21perf08:50:013
1378099230,22cyclictest27599-21idleruntime-cro09:30:003
1378099230,22cyclictest17966-21ls11:30:013
1378099230,22cyclictest14191-21ssh12:31:033
1378099230,22cyclictest11953-21ssh09:47:433
1378099230,0cyclictest0-21swapper/310:53:303
13778992323,0cyclictest7540-21diskmemload11:05:062
13778992323,0cyclictest0-21swapper/211:01:492
13778992323,0cyclictest0-21swapper/210:37:492
13778992323,0cyclictest0-21swapper/209:38:432
13778992322,1cyclictest0-21swapper/211:37:582
13778992322,1cyclictest0-21swapper/210:07:372
13778992322,1cyclictest0-21swapper/209:28:212
13778992321,1cyclictest26312-21ssh10:33:172
13778992321,1cyclictest13112-21perf08:15:012
13778992319,3cyclictest0-21swapper/211:48:262
1377899230,23cyclictest0-21swapper/209:48:262
1377899230,22cyclictest778-21ssh10:40:582
1377899230,22cyclictest10260-21rm11:23:012
1377899230,22cyclictest0-21swapper/211:56:022
1377899230,22cyclictest0-21swapper/211:31:292
1377899230,22cyclictest0-21swapper/210:11:592
1377899230,22cyclictest0-21swapper/209:16:452
1377899230,0cyclictest7540-21diskmemload11:27:082
1377899230,0cyclictest0-21swapper/210:26:432
1377899230,0cyclictest0-21swapper/209:13:512
13773992323,0cyclictest0-21swapper/112:22:421
13773992323,0cyclictest0-21swapper/110:51:561
13773992322,1cyclictest26577-21ssh12:10:501
13773992321,1cyclictest383-21diskstats11:13:011
13773992321,1cyclictest29307-21sh11:41:551
13773992321,1cyclictest0-21swapper/110:59:431
13773992320,2cyclictest24877-21sh11:37:511
1377399230,23cyclictest7540-21diskmemload12:26:281
1377399230,23cyclictest0-21swapper/112:06:081
1377399230,23cyclictest0-21swapper/111:27:441
1377399230,23cyclictest0-21swapper/110:41:041
1377399230,23cyclictest0-21swapper/110:18:211
1377399230,23cyclictest0-21swapper/110:06:281
1377399230,23cyclictest0-21swapper/109:31:421
1377399230,22cyclictest7540-21diskmemload12:39:061
1377399230,22cyclictest24230-21ssh09:27:241
1377399230,22cyclictest16877-21sh12:01:151
1377399230,22cyclictest0-21swapper/111:05:541
1377399230,0cyclictest0-21swapper/111:55:131
1377399230,0cyclictest0-21swapper/110:46:231
1377399230,0cyclictest0-21swapper/109:44:501
1377399230,0cyclictest0-21swapper/109:35:301
13769992323,0cyclictest0-21swapper/012:02:210
13769992322,1cyclictest22988-21diskmemload11:34:590
13769992322,1cyclictest11420-21ssh09:46:520
13769992321,1cyclictest0-21swapper/010:03:000
13769992320,2cyclictest12739-21ssh10:20:010
1376999230,2cyclictest30638-21sh10:05:450
1376999230,23cyclictest7540-21diskmemload09:33:060
1376999230,23cyclictest0-21swapper/009:22:080
1376999230,22cyclictest19192-21dpkg07:20:020
1376999230,22cyclictest15882-21ssh11:59:590
1376999230,22cyclictest0-21swapper/009:18:000
1376999230,1cyclictest27045-21rm11:07:270
1376999230,0cyclictest0-21swapper/011:27:490
1376999230,0cyclictest0-21swapper/009:58:190
13780992222,0cyclictest7540-21diskmemload10:25:463
13780992221,1cyclictest7540-21diskmemload10:20:133
13780992220,1cyclictest15537-21awk08:20:013
13780992220,1cyclictest0-21swapper/310:46:323
13780992220,1cyclictest0-21swapper/308:10:013
13780992220,1cyclictest0-21swapper/307:20:013
13780992219,2cyclictest5059-21sh12:22:173
1378099220,22cyclictest0-21swapper/312:10:403
1378099220,22cyclictest0-21swapper/310:55:263
1378099220,22cyclictest0-21swapper/310:19:423
1378099220,21cyclictest0-21swapper/309:55:403
1378099220,21cyclictest0-21swapper/309:50:523
1378099220,21cyclictest0-21swapper/309:41:303
1378099220,21cyclictest0-21swapper/307:47:583
1378099220,1cyclictest7540-21diskmemload11:36:053
1378099220,0cyclictest0-21swapper/310:33:203
1378099220,0cyclictest0-21swapper/309:16:523
13778992222,0cyclictest7540-21diskmemload12:37:052
13778992222,0cyclictest0-21swapper/210:20:442
13778992222,0cyclictest0-21swapper/210:03:522
13778992220,1cyclictest10718-21munin-run08:10:012
13778992218,4cyclictest0-21swapper/209:56:392
13778992218,0cyclictest0-21swapper/209:52:542
1377899220,21cyclictest7540-21diskmemload10:16:262
1377899220,21cyclictest32724-21munin-run08:55:012
1377899220,21cyclictest18404-21apt-get08:27:592
1377899220,21cyclictest0-21swapper/212:31:462
1377899220,21cyclictest0-21swapper/212:00:372
1377899220,21cyclictest0-21swapper/211:40:392
1377899220,1cyclictest0-21swapper/212:22:552
1377899220,0cyclictest151rcu_preempt12:26:402
1377899220,0cyclictest0-21swapper/209:40:102
13773992222,0cyclictest0-21swapper/111:16:111
13773992220,1cyclictest28645-21cron07:40:001
13773992220,1cyclictest23768-21munin-run07:30:011
13773992219,2cyclictest14594-21ssh12:31:401
1377399220,21cyclictest7540-21diskmemload10:00:271
1377399220,21cyclictest0-21swapper/111:00:011
1377399220,21cyclictest0-21swapper/110:21:051
1377399220,21cyclictest0-21swapper/108:52:591
1377399220,1cyclictest0-21swapper/109:24:541
1377399220,0cyclictest0-21swapper/109:45:341
13769992222,0cyclictest7540-21diskmemload09:27:250
13769992222,0cyclictest0-21swapper/012:05:340
13769992222,0cyclictest0-21swapper/009:35:300
13769992221,1cyclictest11984-21ssh11:23:190
1376999220,22cyclictest0-21swapper/012:28:140
1376999220,22cyclictest0-21swapper/010:11:110
1376999220,21cyclictest7540-21diskmemload11:50:220
1376999220,21cyclictest7540-21diskmemload11:15:510
1376999220,21cyclictest7540-21diskmemload10:51:250
1376999220,21cyclictest0-21swapper/012:20:210
1376999220,21cyclictest0-21swapper/011:38:020
1376999220,21cyclictest0-21swapper/008:18:000
1376999220,1cyclictest20151-21rm12:37:380
1376999220,0cyclictest0-21swapper/011:00:170
1376999220,0cyclictest0-21swapper/010:55:550
13780992120,1cyclictest0-21swapper/310:00:373
1378099210,21cyclictest7540-21diskmemload09:35:083
1378099210,21cyclictest0-21swapper/311:15:103
1378099210,20cyclictest0-21swapper/308:22:593
1378099210,0cyclictest0-21swapper/307:57:583
13778992120,1cyclictest7540-21diskmemload12:05:322
13778992120,0cyclictest758-21apt-get08:57:592
13778992119,1cyclictest29919-21dump-pmu-power08:50:012
1377899210,20cyclictest0-21swapper/208:02:582
1377899210,0cyclictest0-21swapper/208:37:592
13773992121,0cyclictest0-21swapper/107:57:581
13773992121,0cyclictest0-21swapper/107:22:591
13773992120,0cyclictest28007-21apt-get08:47:591
13773992119,1cyclictest11187-21apt-get08:12:591
13769992121,0cyclictest0-21swapper/008:12:590
13769992118,2cyclictest8300-21dump-pmu-power08:05:010
1376999210,21cyclictest0-21swapper/012:15:470
1376999210,1cyclictest7540-21diskmemload10:25:240
1376999210,1cyclictest18536-21idleruntime-cro07:20:010
1376999210,0cyclictest0-21swapper/009:02:590
13780992020,0cyclictest0-21swapper/307:27:593
13780992018,1cyclictest29157-21apt-get07:42:593
1378099200,0cyclictest0-21swapper/309:08:013
13778992020,0cyclictest0-21swapper/207:42:592
13778992020,0cyclictest0-21swapper/207:32:582
13778992019,1cyclictest0-21swapper/207:47:582
13773992020,0cyclictest0-21swapper/108:57:591
13773992017,2cyclictest18530-21cron07:20:011
1377399200,0cyclictest0-21swapper/107:47:581
13769992019,1cyclictest3927-21apt-get07:57:580
13769992018,1cyclictest31061-21apt07:45:000
13780991919,0cyclictest0-21swapper/308:37:583
13780991919,0cyclictest0-21swapper/307:32:583
13778991919,0cyclictest0-21swapper/208:32:582
13778991919,0cyclictest0-21swapper/208:18:002
13778991919,0cyclictest0-21swapper/207:22:592
13773991919,0cyclictest16009-21apt-get08:22:591
13773991919,0cyclictest0-21swapper/109:08:011
13769991919,0cyclictest0-21swapper/008:57:590
13769991919,0cyclictest0-21swapper/008:08:000
13769991919,0cyclictest0-21swapper/007:32:580
13769991917,1cyclictest0-21swapper/008:55:010
13780991817,1cyclictest11187-21apt-get08:12:593
13780991817,0cyclictest0-21swapper/307:20:023
13780991816,1cyclictest32681-21cron08:55:013
13778991818,0cyclictest0-21swapper/209:08:012
13778991818,0cyclictest0-21swapper/208:42:592
13778991817,0cyclictest0-21swapper/209:02:592
13778991816,1cyclictest3438-21dump-pmu-power07:55:002
13778991816,1cyclictest17933-21munin-run08:25:002
13773991818,0cyclictest24241-21apt-get07:32:581
13773991818,0cyclictest1520-21apt-get07:52:591
13773991818,0cyclictest0-21swapper/107:42:591
13780991716,0cyclictest0-21swapper/307:52:593
13778991716,0cyclictest0-21swapper/207:37:592
13773991717,0cyclictest0-21swapper/108:27:591
1377399170,16cyclictest313-21systemd-journal07:10:191
13780991616,0cyclictest6398-21apt-get08:02:583
13780991616,0cyclictest0-21swapper/307:12:583
13778991614,1cyclictest21846-21apt-get07:27:592
13773991616,0cyclictest0-21swapper/109:02:591
13773991616,0cyclictest0-21swapper/108:37:581
13769991616,0cyclictest31547-21apt-get07:47:580
13769991616,0cyclictest0-21swapper/008:37:580
13769991616,0cyclictest0-21swapper/008:35:000
13769991615,1cyclictest5648-21apt-get09:08:000
13780991515,0cyclictest0-21swapper/307:38:003
1378099150,1cyclictest4999-21taskset09:03:193
13769991515,0cyclictest0-21swapper/008:42:590
1376999150,1cyclictest19310-21memory08:28:040
13780991414,0cyclictest0-21swapper/308:40:533
1378099140,14cyclictest0-21swapper/308:30:583
1378099140,14cyclictest0-21swapper/308:26:293
1378099140,0cyclictest0-21swapper/308:55:053
1377899140,14cyclictest0-21swapper/207:55:302
1377899140,14cyclictest0-21swapper/207:15:282
1377899140,0cyclictest0-21swapper/207:11:162
13773991414,0cyclictest0-21swapper/108:30:151
1377399140,14cyclictest0-21swapper/108:40:241
1377399140,0cyclictest0-21swapper/108:05:241
13769991414,0cyclictest0-21swapper/008:20:080
13769991414,0cyclictest0-21swapper/007:35:340
1376999140,14cyclictest0-21swapper/007:10:140
1376999140,0cyclictest0-21swapper/008:45:090
1376999140,0cyclictest0-21swapper/007:50:010
1376999140,0cyclictest0-21swapper/007:26:550
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional