You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 09:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sat Jul 04, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26583992713,13cyclictest0-21swapper/323:56:163
2657699270,1cyclictest12089-21hddtemp_smartct23:56:191
26576992622,3cyclictest20204-21diskmemload23:44:091
26576992620,5cyclictest21333-21munin-run21:40:011
2657699260,25cyclictest16609-21ssh00:29:011
2657699260,1cyclictest5266-21ssh00:18:551
2657699260,1cyclictest0-21swapper/123:17:031
2657299260,25cyclictest17935-21diskmemload23:31:430
2658399250,24cyclictest23068-21apt-get00:06:183
26576992523,2cyclictest0-21swapper/123:52:091
26576992523,2cyclictest0-21swapper/122:08:331
26576992522,2cyclictest13810-21ssh21:32:291
26576992520,4cyclictest7583-21sh22:54:531
26576992520,4cyclictest20204-21diskmemload23:05:361
2657699250,24cyclictest28566-21turbostat00:40:011
2657699250,21cyclictest11102-21sh00:24:131
2657699250,1cyclictest0-21swapper/123:27:431
26572992524,0cyclictest0-21swapper/023:39:120
2657299250,24cyclictest4876-21ssh21:25:210
2657299250,24cyclictest26217-21sh22:12:540
2657299250,24cyclictest20204-21diskmemload23:57:580
26583992423,1cyclictest0-21swapper/322:06:453
26583992422,1cyclictest0-21swapper/322:01:263
2658399240,24cyclictest0-21swapper/322:38:543
2658399240,23cyclictest5477-21turbostat.cron21:55:003
2658399240,23cyclictest20204-21diskmemload00:15:543
2658399240,23cyclictest0-21swapper/323:43:243
2658399240,23cyclictest0-21swapper/323:30:063
2658399240,23cyclictest0-21swapper/322:23:423
2658399240,0cyclictest0-21swapper/300:03:083
26578992422,1cyclictest7043-21ssh22:25:252
26578992420,3cyclictest412-21ssh22:19:272
2657899240,24cyclictest0-21swapper/223:24:082
2657899240,23cyclictest26832-21ssh00:37:362
2657899240,23cyclictest25346-21ssh21:43:102
2657899240,23cyclictest20204-21diskmemload21:53:222
2657899240,0cyclictest20204-21diskmemload23:09:092
2657899240,0cyclictest0-21swapper/222:06:552
26576992421,3cyclictest0-21swapper/123:46:211
26576992421,3cyclictest0-21swapper/121:50:091
26576992421,3cyclictest0-21swapper/121:49:001
2657699240,24cyclictest20204-21diskmemload21:19:341
2657699240,24cyclictest0-21swapper/123:22:481
2657699240,24cyclictest0-21swapper/122:24:411
2657699240,23cyclictest0-21swapper/123:10:211
2657699240,23cyclictest0-21swapper/122:56:221
2657699240,23cyclictest0-21swapper/122:31:591
2657699240,23cyclictest0-21swapper/121:55:081
2657699240,21cyclictest0-21swapper/100:10:341
2657699240,0cyclictest0-21swapper/122:35:321
2657699240,0cyclictest0-21swapper/122:16:541
2657699240,0cyclictest0-21swapper/121:25:551
2657699240,0cyclictest0-21swapper/100:01:131
26572992423,1cyclictest0-21swapper/023:27:330
26572992423,0cyclictest0-21swapper/000:04:260
2657299241,2cyclictest12103-21cat22:30:000
2657299240,24cyclictest0-21swapper/021:49:500
2657299240,23cyclictest20204-21diskmemload21:10:260
2657299240,23cyclictest14652-21cstates23:01:160
2657299240,23cyclictest0-21swapper/021:55:160
2657299240,1cyclictest6497-21latency_hist22:25:010
2657299240,1cyclictest20204-21diskmemload23:48:450
2657299240,0cyclictest13443-21ssh22:01:230
2657299240,0cyclictest0-21swapper/023:22:020
2657299240,0cyclictest0-21swapper/021:41:550
26583992323,0cyclictest0-21swapper/322:57:183
26583992323,0cyclictest0-21swapper/322:31:413
26583992323,0cyclictest0-21swapper/321:44:383
26583992323,0cyclictest0-21swapper/300:25:443
26583992322,1cyclictest20204-21diskmemload22:50:083
26583992322,0cyclictest0-21swapper/300:11:313
26583992321,1cyclictest23999-21ssh00:36:143
26583992321,1cyclictest0-21swapper/323:00:053
26583992321,1cyclictest0-21swapper/321:58:253
26583992321,1cyclictest0-21swapper/321:46:173
26583992321,1cyclictest0-21swapper/321:28:103
26583992321,1cyclictest0-21swapper/321:19:143
2658399230,23cyclictest0-21swapper/322:12:543
2658399230,23cyclictest0-21swapper/321:39:083
2658399230,23cyclictest0-21swapper/321:32:253
2658399230,23cyclictest0-21swapper/321:10:263
2658399230,22cyclictest7712-21ssh23:51:283
2658399230,22cyclictest7605-21ssh22:26:173
2658399230,22cyclictest1623-21grep23:46:253
2658399230,22cyclictest0-21swapper/323:35:413
2658399230,22cyclictest0-21swapper/300:30:453
2658399230,0cyclictest20204-21diskmemload23:26:013
2658399230,0cyclictest0-21swapper/323:22:023
2658399230,0cyclictest0-21swapper/323:19:513
26578992323,0cyclictest0-21swapper/200:12:112
26578992322,1cyclictest0-21swapper/222:20:082
26578992321,1cyclictest0-21swapper/223:00:572
26578992320,2cyclictest19603-21ssh00:02:282
2657899230,23cyclictest20204-21diskmemload23:53:542
2657899230,23cyclictest0-21swapper/223:48:152
2657899230,23cyclictest0-21swapper/222:51:062
2657899230,23cyclictest0-21swapper/221:15:422
2657899230,22cyclictest8281-21ssh22:55:262
2657899230,22cyclictest0-21swapper/222:45:482
2657899230,21cyclictest313-21systemd-journal19:25:002
2657899230,0cyclictest0-21swapper/223:42:152
2657899230,0cyclictest0-21swapper/223:36:402
2657899230,0cyclictest0-21swapper/222:43:162
2657899230,0cyclictest0-21swapper/222:04:322
2657899230,0cyclictest0-21swapper/221:56:582
2657899230,0cyclictest0-21swapper/200:06:442
26576992322,1cyclictest20204-21diskmemload23:00:021
26576992321,1cyclictest25756-21sh20:15:001
26576992320,3cyclictest0-21swapper/122:44:211
26576992318,4cyclictest20204-21diskmemload00:06:081
2657699231,21cyclictest20989-21wc20:05:011
2657699230,23cyclictest0-21swapper/123:35:131
2657699230,23cyclictest0-21swapper/123:31:371
2657699230,20cyclictest20204-21diskmemload21:10:291
2657699230,20cyclictest0-21swapper/122:14:541
26572992323,0cyclictest0-21swapper/022:50:080
26572992323,0cyclictest0-21swapper/021:51:230
26572992323,0cyclictest0-21swapper/000:23:570
26572992323,0cyclictest0-21swapper/000:11:310
26572992323,0cyclictest0-21swapper/000:05:030
26572992322,1cyclictest0-21swapper/000:32:200
26572992321,1cyclictest0-21swapper/023:19:410
26572992321,1cyclictest0-21swapper/022:40:210
26572992320,2cyclictest28623-21sh22:15:530
2657299230,2cyclictest16862-21ssh22:05:020
2657299230,2cyclictest1535-21sh00:16:030
2657299230,23cyclictest0-21swapper/022:35:460
2657299230,23cyclictest0-21swapper/022:31:410
2657299230,23cyclictest0-21swapper/000:25:440
2657299230,22cyclictest26492-21ssh21:15:500
2657299230,22cyclictest0-21swapper/022:46:240
2657299230,21cyclictest0-21swapper/021:36:490
2657299230,1cyclictest20204-21diskmemload21:31:380
2657299230,1cyclictest0-21swapper/000:39:420
26583992220,1cyclictest32096-21ssh22:46:473
26583992220,1cyclictest0-21swapper/320:56:173
2658399220,21cyclictest23387-21munin-run20:10:013
2658399220,21cyclictest0-21swapper/323:10:243
2658399220,21cyclictest0-21swapper/321:21:183
2658399220,21cyclictest0-21swapper/320:05:013
2658399220,21cyclictest0-21swapper/319:51:163
2658399220,21cyclictest0-21swapper/319:51:163
2658399220,21cyclictest0-21swapper/319:26:163
2658399220,21cyclictest0-21swapper/300:22:063
2658399220,1cyclictest20212-21dump-pmu-power21:10:003
2658399220,1cyclictest0-21swapper/322:15:463
2658399220,0cyclictest20204-21diskmemload22:44:033
26578992222,0cyclictest0-21swapper/223:11:572
26578992222,0cyclictest0-21swapper/221:26:002
26578992222,0cyclictest0-21swapper/200:15:392
26578992220,1cyclictest0-21swapper/200:30:222
26578992219,0cyclictest0-21swapper/221:32:232
2657899221,21cyclictest20204-21diskmemload00:20:412
2657899220,22cyclictest0-21swapper/223:56:442
2657899220,21cyclictest3755-21latency_hist19:30:002
2657899220,21cyclictest26120-21apt-get20:16:162
2657899220,21cyclictest17438-21apt-get21:05:022
2657899220,21cyclictest0-21swapper/223:25:462
2657899220,21cyclictest0-21swapper/221:20:182
2657899220,21cyclictest0-21swapper/200:25:282
2657899220,1cyclictest3503-21perf23:20:002
2657899220,1cyclictest18555-21ssh23:32:402
2657899220,1cyclictest18300-21ssh22:35:362
26576992221,1cyclictest11830-21sh22:29:411
26576992221,1cyclictest0-21swapper/121:41:161
26576992221,1cyclictest0-21swapper/119:55:011
26576992221,1cyclictest0-21swapper/119:55:011
26576992220,1cyclictest12977-21apt-get20:56:181
26576992217,4cyclictest17988-21ssh00:30:371
2657699220,2cyclictest1379-21munin-plugin-st19:25:001
2657699220,1cyclictest0-21swapper/119:16:161
26572992222,0cyclictest20204-21diskmemload23:41:570
26572992220,1cyclictest7041-21ssh23:51:240
2657299220,21cyclictest18598-21munin-run20:00:010
2657299220,21cyclictest1721-21apt-get19:26:160
2657299220,21cyclictest0-21swapper/023:05:290
2657299220,21cyclictest0-21swapper/021:01:170
2657299220,21cyclictest0-21swapper/020:30:000
2657299220,21cyclictest0-21swapper/019:21:160
2657299220,1cyclictest0-21swapper/023:10:270
2657299220,1cyclictest0-21swapper/021:20:350
2657299220,0cyclictest0-21swapper/022:56:160
26583992121,0cyclictest31689-21apt-get19:21:163
2658399210,20cyclictest0-21swapper/323:05:293
2658399210,0cyclictest0-21swapper/320:26:173
26578992121,0cyclictest0-21swapper/220:51:182
26578992119,1cyclictest313-21systemd-journal19:35:002
2657899210,21cyclictest0-21swapper/221:10:472
2657899210,20cyclictest20204-21diskmemload21:35:262
2657699210,0cyclictest0-21swapper/120:31:161
2657699210,0cyclictest0-21swapper/120:21:161
26572992121,0cyclictest0-21swapper/020:56:170
26572992121,0cyclictest0-21swapper/019:46:170
2657299210,20cyclictest10582-21apt-get20:51:180
2657299210,20cyclictest0-21swapper/020:36:170
2657299210,0cyclictest0-21swapper/020:01:170
26583992020,0cyclictest0-21swapper/320:51:183
26583992020,0cyclictest0-21swapper/319:36:163
26583992019,1cyclictest28545-21apt-get20:21:163
2658399200,0cyclictest16518-21apt-get19:56:163
2658399200,0cyclictest0-21swapper/320:16:163
26578992020,0cyclictest0-21swapper/220:41:182
26578992020,0cyclictest0-21swapper/220:31:162
26578992018,1cyclictest18536-21cron20:00:012
26576992020,0cyclictest0-21swapper/120:36:171
26576992020,0cyclictest0-21swapper/119:46:171
26576992013,6cyclictest6169-21grep19:35:001
2657699200,0cyclictest0-21swapper/120:06:161
26572992018,1cyclictest6519-21apt-get19:36:160
26572992018,1cyclictest0-21swapper/021:05:020
2657299200,19cyclictest0-21swapper/020:21:160
2657299200,0cyclictest7618-21kworker/0:020:15:580
26576991919,0cyclictest0-21swapper/121:06:161
26576991919,0cyclictest0-21swapper/119:41:171
26576991919,0cyclictest0-21swapper/119:26:161
26583991816,1cyclictest0-21swapper/321:01:173
26578991818,0cyclictest0-21swapper/220:01:172
26578991817,1cyclictest583-21perf20:30:002
26578991817,0cyclictest15394-21apt-get21:01:172
26578991816,1cyclictest8596-21munin-run19:40:002
26578991816,1cyclictest28928-21munin-run19:15:012
2657899180,17cyclictest0-21swapper/219:46:172
2657699185,3cyclictest8188-21apt-get20:46:171
26576991818,0cyclictest0-21swapper/120:41:171
26576991818,0cyclictest0-21swapper/119:36:161
2658399170,2cyclictest10995-21perf19:45:003
26578991717,0cyclictest3373-21apt-get20:36:162
26578991717,0cyclictest0-21swapper/219:51:162
26578991717,0cyclictest0-21swapper/219:51:162
26576991717,0cyclictest0-21swapper/120:26:171
26576991716,0cyclictest0-21swapper/120:51:181
26572991716,1cyclictest23384-21perf20:10:010
26583991616,0cyclictest0-21swapper/320:36:163
26578991616,0cyclictest0-21swapper/220:56:182
26578991616,0cyclictest0-21swapper/220:11:172
26578991614,1cyclictest0-21swapper/220:06:252
2657899160,0cyclictest0-21swapper/219:41:172
26576991614,1cyclictest0-21swapper/119:56:161
2657699160,15cyclictest26758-21tail20:16:191
26572991614,1cyclictest0-21swapper/020:41:200
26572991614,1cyclictest0-21swapper/019:51:220
26572991614,1cyclictest0-21swapper/019:51:220
26572991614,1cyclictest0-21swapper/019:16:200
26583991514,1cyclictest0-21swapper/319:15:013
26583991513,1cyclictest6431-21gltestperf20:41:203
2658399150,14cyclictest0-21swapper/319:31:223
26578991515,0cyclictest0-21swapper/219:16:162
2658399140,14cyclictest0-21swapper/320:31:513
2658399140,0cyclictest0-21swapper/320:45:503
2658399140,0cyclictest0-21swapper/320:11:483
2658399140,0cyclictest0-21swapper/319:15:013
26578991414,0cyclictest0-21swapper/220:46:172
2657899140,14cyclictest0-21swapper/220:21:092
2657699140,0cyclictest0-21swapper/121:00:001
26572991414,0cyclictest0-21swapper/020:45:210
26572991414,0cyclictest0-21swapper/019:41:160
2657299140,14cyclictest0-21swapper/020:30:010
2657299140,14cyclictest0-21swapper/020:10:140
2657299140,14cyclictest0-21swapper/019:30:320
2657299140,14cyclictest0-21swapper/019:10:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional