You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 21:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Fri May 15, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
928699280,2cyclictest13429-21awk08:25:011
928699280,24cyclictest12626-21unixbench_multi10:42:211
928799270,26cyclictest8222-21ssh10:38:592
9286992722,4cyclictest3054-21diskmemload12:24:461
9287992619,6cyclictest3054-21diskmemload10:52:212
928799260,25cyclictest6812-21ssh10:09:482
928799260,25cyclictest22198-21apt-get07:37:102
928799260,25cyclictest0-21swapper/208:22:122
928799260,22cyclictest7301-21ssh10:10:142
928799260,22cyclictest3724-21rm12:00:302
9286992621,4cyclictest4354-21ssh10:36:041
928699260,25cyclictest0-21swapper/107:17:121
928699260,25cyclictest0-21swapper/107:17:121
928699260,1cyclictest3297-21munin-run10:35:011
928699260,1cyclictest13334-21apt-get10:15:021
928699260,1cyclictest0-21swapper/111:37:231
928699260,1cyclictest0-21swapper/109:13:581
928899250,24cyclictest12641-21cat09:47:123
928899250,1cyclictest0-21swapper/310:17:203
9287992523,1cyclictest3388-21df_inode11:32:122
9287992520,4cyclictest0-21swapper/209:35:582
928799250,24cyclictest21030-21idleruntime-cro08:40:002
928799250,24cyclictest0-21swapper/210:48:092
928799250,21cyclictest0-21swapper/210:32:482
928799250,1cyclictest10659-21apt-get09:17:132
928799250,1cyclictest0-21swapper/212:22:172
928799250,1cyclictest0-21swapper/212:09:382
928799250,0cyclictest0-21swapper/211:22:132
9286992522,3cyclictest0-21swapper/109:23:111
928699250,2cyclictest2008-21ssh09:37:161
928699250,24cyclictest3953-21ssh11:03:371
928699250,24cyclictest0-21swapper/109:48:411
928699250,1cyclictest3992-21ssh12:00:501
928699250,1cyclictest32132-21ssh11:57:151
928699250,1cyclictest21429-21ssh09:54:331
928699250,1cyclictest16179-21ssh11:14:461
928699250,1cyclictest0-21swapper/111:51:001
928699250,1cyclictest0-21swapper/110:29:311
928599250,24cyclictest3054-21diskmemload10:10:170
9288992423,1cyclictest3054-21diskmemload11:25:573
9288992423,1cyclictest10099-21ssh10:41:083
928899242,21cyclictest3054-21diskmemload11:06:043
9288992420,3cyclictest18171-21apt11:45:013
928899240,24cyclictest0-21swapper/311:39:003
928899240,23cyclictest6387-21ssh09:41:583
928899240,23cyclictest3054-21diskmemload11:12:363
928899240,23cyclictest3054-21diskmemload10:58:223
928899240,1cyclictest16207-21sh08:30:003
9287992420,3cyclictest3054-21diskmemload11:37:112
9287992420,3cyclictest0-21swapper/209:24:052
9287992419,4cyclictest0-21swapper/209:57:122
928799241,22cyclictest0-21swapper/212:40:002
928799240,23cyclictest3054-21diskmemload09:40:142
928799240,23cyclictest10854-21apt-get10:42:112
928799240,23cyclictest0-21swapper/212:16:272
928799240,23cyclictest0-21swapper/211:46:472
928799240,23cyclictest0-21swapper/210:25:122
928799240,23cyclictest0-21swapper/209:27:542
928799240,20cyclictest0-21swapper/209:51:002
928799240,0cyclictest0-21swapper/210:17:122
9286992421,2cyclictest606-21dbus-daemon10:25:001
9286992421,2cyclictest3054-21diskmemload09:59:151
9286992420,3cyclictest0-21swapper/111:45:241
9286992420,3cyclictest0-21swapper/111:16:011
9286992419,4cyclictest3054-21diskmemload12:15:361
928699240,24cyclictest0-21swapper/110:12:061
928699240,1cyclictest0-21swapper/112:34:181
928699240,0cyclictest0-21swapper/112:25:101
928699240,0cyclictest0-21swapper/112:05:391
9285992423,1cyclictest0-21swapper/010:19:050
928599242,22cyclictest62550irq/127-eno111:00:160
928599240,23cyclictest3054-21diskmemload10:37:260
928599240,22cyclictest5900-21sh11:33:510
928599240,0cyclictest3054-21diskmemload10:55:090
928599240,0cyclictest0-21swapper/012:24:270
928599240,0cyclictest0-21swapper/012:14:440
9288992323,0cyclictest0-21swapper/310:00:213
9288992321,1cyclictest26684-21ssh09:58:483
9288992321,1cyclictest0-21swapper/312:00:283
9288992321,1cyclictest0-21swapper/311:00:163
928899230,23cyclictest3054-21diskmemload11:49:513
928899230,22cyclictest30628-21sh12:23:113
928899230,22cyclictest29655-21ssh09:32:243
928899230,22cyclictest21120-21ssh11:18:513
928899230,22cyclictest21033-21ssh12:15:253
928899230,22cyclictest0-21swapper/309:26:143
928899230,1cyclictest4783-21sh11:32:213
928899230,0cyclictest0-21swapper/312:36:373
928899230,0cyclictest0-21swapper/312:34:283
928899230,0cyclictest0-21swapper/310:36:443
928899230,0cyclictest0-21swapper/310:12:183
9287992323,0cyclictest0-21swapper/210:55:162
9287992322,1cyclictest3054-21diskmemload09:45:552
9287992322,1cyclictest0-21swapper/211:40:282
9287992321,2cyclictest0-21swapper/212:34:192
9287992321,2cyclictest0-21swapper/212:25:122
9287992321,2cyclictest0-21swapper/210:20:062
9287992320,3cyclictest0-21swapper/211:58:412
9287992320,0cyclictest0-21swapper/212:10:072
928799230,23cyclictest0-21swapper/211:26:332
928799230,23cyclictest0-21swapper/209:10:492
928799230,22cyclictest26979-21ssh09:31:152
928799230,20cyclictest0-21swapper/211:16:232
928799230,20cyclictest0-21swapper/211:02:032
928799230,0cyclictest0-21swapper/211:06:472
928799230,0cyclictest0-21swapper/210:00:252
9286992323,0cyclictest0-21swapper/111:22:561
9286992323,0cyclictest0-21swapper/111:05:081
9286992323,0cyclictest0-21swapper/110:07:101
9286992323,0cyclictest0-21swapper/109:27:241
9286992320,3cyclictest0-21swapper/111:26:201
9286992320,2cyclictest0-21swapper/109:18:471
9286992319,3cyclictest14908-21ssh12:37:491
928699230,2cyclictest26709-21ssh09:30:441
928699230,23cyclictest0-21swapper/110:03:121
928699230,22cyclictest5816-21ssh09:41:041
928699230,22cyclictest27882-21rm10:57:081
928699230,22cyclictest13867-21rm11:41:431
928699230,20cyclictest0-21swapper/112:10:151
928699230,1cyclictest313-21systemd-journal07:45:001
928699230,0cyclictest3054-21diskmemload10:46:531
928699230,0cyclictest0-21swapper/111:32:411
928699230,0cyclictest0-21swapper/110:50:121
9285992323,0cyclictest3054-21diskmemload09:15:080
9285992323,0cyclictest0-21swapper/011:56:060
9285992323,0cyclictest0-21swapper/011:25:140
9285992322,0cyclictest0-21swapper/008:17:110
9285992321,1cyclictest21685-21sh12:16:130
9285992321,1cyclictest0-21swapper/010:51:240
9285992321,1cyclictest0-21swapper/010:05:310
9285992321,1cyclictest0-21swapper/009:42:110
9285992320,2cyclictest17817-21rm09:51:560
928599230,2cyclictest11057-21sh11:10:140
928599230,23cyclictest0-21swapper/012:09:150
928599230,23cyclictest0-21swapper/012:00:280
928599230,23cyclictest0-21swapper/011:40:470
928599230,22cyclictest9795-21sh10:40:450
928599230,22cyclictest62550irq/127-eno112:28:300
928599230,22cyclictest22407-21rm09:55:270
928599230,22cyclictest19846-21taskset10:21:160
928599230,22cyclictest12011-21ssh09:46:440
928599230,22cyclictest0-21swapper/012:31:140
928599230,22cyclictest0-21swapper/011:45:350
928599230,22cyclictest0-21swapper/010:31:350
928599230,22cyclictest0-21swapper/010:29:070
928599230,22cyclictest0-21swapper/010:02:340
928599230,22cyclictest0-21swapper/009:37:060
928599230,21cyclictest0-21swapper/009:24:400
928599230,1cyclictest0-21swapper/010:47:430
9288992222,0cyclictest0-21swapper/311:24:583
9288992222,0cyclictest0-21swapper/310:28:063
9288992220,1cyclictest24747-21ssh11:51:063
9288992220,1cyclictest1855-21apt-get08:02:113
9288992219,2cyclictest15992-21sh12:11:153
928899220,22cyclictest0-21swapper/312:07:343
928899220,21cyclictest833-21apt-get10:05:023
928899220,21cyclictest30665-21ssh10:30:333
928899220,21cyclictest3054-21diskmemload09:50:163
928899220,21cyclictest2469-21nvmesmart_nvme012:27:163
928899220,21cyclictest15110-21ssh10:45:103
928899220,21cyclictest0-21swapper/311:55:103
928899220,21cyclictest0-21swapper/309:20:413
928899220,1cyclictest3598-21ssh09:10:223
928899220,1cyclictest13610-21sh09:18:523
928899220,1cyclictest11681-21munin-run07:15:003
928899220,0cyclictest0-21swapper/310:20:173
928899220,0cyclictest0-21swapper/309:38:253
928899220,0cyclictest0-21swapper/307:37:103
9287992220,1cyclictest25185-21ssh11:51:392
9287992216,1cyclictest14039-21cron07:20:012
9287992216,1cyclictest14039-21cron07:20:012
928799220,21cyclictest16656-21apt-get08:32:112
928799220,21cyclictest0-21swapper/208:12:122
928799220,0cyclictest0-21swapper/208:27:112
928699220,21cyclictest14257-21apt-get08:27:111
9285992220,1cyclictest0-21swapper/009:30:080
928599221,0cyclictest62550irq/127-eno109:12:220
928599220,21cyclictest3054-21diskmemload12:35:370
928599220,21cyclictest3054-21diskmemload09:25:130
928599220,21cyclictest0-21swapper/011:51:180
928599220,21cyclictest0-21swapper/011:06:340
928599220,1cyclictest7861-21ssh11:36:220
928599220,1cyclictest3054-21diskmemload11:16:040
9288992120,0cyclictest24615-21apt-get07:42:113
9288992120,0cyclictest0-21swapper/308:22:113
928899210,21cyclictest3054-21diskmemload10:50:133
928899210,1cyclictest27063-21apt-get07:47:113
928899210,1cyclictest21672-21grep07:35:013
9287992120,1cyclictest14907-21apt-get07:22:122
9287992120,0cyclictest21470-21apt-get08:42:122
9286992121,0cyclictest0-21swapper/107:20:021
9286992119,1cyclictest606-21dbus-daemon08:45:001
928699210,21cyclictest0-21swapper/107:47:111
928699210,1cyclictest28279-21apt08:55:001
9285992121,0cyclictest0-21swapper/008:42:120
9285992121,0cyclictest0-21swapper/007:22:120
9285992121,0cyclictest0-21swapper/007:12:130
928599210,20cyclictest17305-21apt-get07:27:120
928599210,20cyclictest0-21swapper/008:12:120
9288992020,0cyclictest12094-21apt-get07:17:123
9288992020,0cyclictest12094-21apt-get07:17:123
9288992020,0cyclictest0-21swapper/308:47:113
9288992020,0cyclictest0-21swapper/308:32:113
9288992020,0cyclictest0-21swapper/307:22:113
928899200,20cyclictest0-21swapper/308:42:123
928899200,20cyclictest0-21swapper/307:27:123
928899200,0cyclictest0-21swapper/308:17:113
9287992019,1cyclictest31125-21apt-get09:02:122
9287992018,1cyclictest28282-21grep08:55:002
928799200,0cyclictest23863-21apt-get08:47:112
9286992018,1cyclictest0-21swapper/108:40:001
928699200,1cyclictest756-21grep07:57:181
928699200,1cyclictest6646-21apt-get08:12:121
928699200,1cyclictest10340-21smartctl07:12:151
9285992020,0cyclictest0-21swapper/007:57:110
9285992020,0cyclictest0-21swapper/007:52:110
9285992019,0cyclictest0-21swapper/008:57:120
928599200,20cyclictest0-21swapper/008:27:110
928599200,20cyclictest0-21swapper/008:02:110
928599200,0cyclictest0-21swapper/008:32:110
9288991919,0cyclictest0-21swapper/308:12:123
9288991919,0cyclictest0-21swapper/308:07:093
9288991919,0cyclictest0-21swapper/307:52:113
9287991919,0cyclictest0-21swapper/207:27:132
9287991918,0cyclictest29457-21apt-get07:52:112
9287991917,1cyclictest1091-21apt-get09:07:102
9286991919,0cyclictest0-21swapper/109:07:101
9286991919,0cyclictest0-21swapper/108:02:111
928699190,1cyclictest0-21swapper/108:10:011
9285991919,0cyclictest0-21swapper/007:17:110
9285991919,0cyclictest0-21swapper/007:17:110
9287991818,0cyclictest28706-21apt-get08:57:122
9287991818,0cyclictest0-21swapper/208:02:112
9287991818,0cyclictest0-21swapper/207:47:112
9286991818,0cyclictest0-21swapper/108:57:121
9286991817,0cyclictest0-21swapper/108:32:111
9285991818,0cyclictest0-21swapper/008:37:110
9285991817,1cyclictest19701-21apt-get07:32:110
9287991716,0cyclictest4247-21apt-get08:07:092
9286991717,0cyclictest0-21swapper/107:27:131
9286991713,3cyclictest0-21swapper/109:02:131
9285991716,1cyclictest24614-21apt07:42:110
9288991616,0cyclictest0-21swapper/308:57:123
9288991616,0cyclictest0-21swapper/307:57:113
9288991614,1cyclictest0-21swapper/309:05:163
9287991614,1cyclictest11057-21awk08:20:012
928699163,12cyclictest22188-21acpi07:37:101
928699160,1cyclictest0-21swapper/108:17:171
9285991616,0cyclictest0-21swapper/007:37:100
9285991614,1cyclictest1091-21apt-get09:07:100
928599160,15cyclictest25271-21switchtime08:47:190
9288991513,1cyclictest0-21swapper/308:52:143
9287991515,0cyclictest31852-21apt-get07:57:112
9286991515,0cyclictest0-21swapper/107:32:121
928899140,0cyclictest0-21swapper/309:00:043
928899140,0cyclictest0-21swapper/308:35:193
928799142,8cyclictest0-21swapper/207:12:122
928799140,14cyclictest0-21swapper/207:40:382
928799140,14cyclictest0-21swapper/207:30:092
928699140,0cyclictest0-21swapper/107:50:501
928599140,14cyclictest0-21swapper/009:00:140
928599140,14cyclictest0-21swapper/007:45:470
928599140,0cyclictest0-21swapper/008:20:540
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional