You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 04:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Wed Jan 07, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1668992825,2cyclictest13-21ksoftirqd/000:00:010
1668992621,4cyclictest22519-21awk21:00:010
1682992422,1cyclictest5679-21cron20:25:003
167799240,23cyclictest13083-21munin-run00:00:012
1673992422,1cyclictest8272-21munin-plugin-st23:50:011
1668992422,1cyclictest0-21swapper/000:11:210
1668992421,2cyclictest13262-21apt20:41:220
166899240,23cyclictest9022-21munin-run22:45:000
1682992323,0cyclictest0-21swapper/321:05:023
168299231,21cyclictest990-21apt-get23:36:203
168299230,22cyclictest3312-21basename20:20:003
168299230,22cyclictest0-21swapper/322:26:203
168299230,1cyclictest6051-21apt-get20:26:203
167799230,22cyclictest0-21swapper/222:41:232
1673992321,1cyclictest13213-21apachectl00:00:011
167399231,1cyclictest27296-21apt-get22:16:221
167399230,2cyclictest21364-21runrttasks23:11:221
167399230,22cyclictest24545-21perf22:10:001
167399230,22cyclictest0-21swapper/121:36:221
166899230,22cyclictest13-21ksoftirqd/021:31:190
166899230,22cyclictest13-21ksoftirqd/021:31:190
1682992222,0cyclictest0-21swapper/323:56:223
1682992220,1cyclictest4018-21idleruntime-cro19:15:013
1682992220,1cyclictest27938-21apt00:30:003
1682992219,2cyclictest25148-21cron00:25:003
168299220,21cyclictest25666-21lockfile-remove20:00:013
168299220,21cyclictest10844-21apt-get20:36:203
168299220,21cyclictest0-21swapper/300:05:013
1677992220,1cyclictest0-21swapper/220:11:232
1677992220,1cyclictest0-21swapper/200:25:022
1677992219,2cyclictest25791-21cron23:20:012
167799220,21cyclictest29411-21ls22:20:012
167799220,21cyclictest11596-21apt-get19:31:202
167799220,21cyclictest0-21swapper/220:51:212
1673992220,1cyclictest14962-21munin-run21:50:011
1673992220,1cyclictest0-21swapper/120:46:221
1673992219,2cyclictest16199-21dump-pmu-power23:00:001
1673992219,2cyclictest12913-21idleruntime-cro20:40:011
1673992218,3cyclictest32163-21apt-get22:26:201
167399220,21cyclictest29768-21apt-get22:21:201
167399220,21cyclictest25174-21munin-run00:25:001
167399220,21cyclictest20447-21apt-get20:56:221
167399220,21cyclictest0-21swapper/123:16:221
167399220,21cyclictest0-21swapper/122:11:221
167399220,21cyclictest0-21swapper/122:01:211
167399220,21cyclictest0-21swapper/121:56:221
167399220,21cyclictest0-21swapper/121:51:211
1668992220,1cyclictest9370-21apt-get22:46:220
1668992219,1cyclictest313-21systemd-journal22:15:010
166899220,2cyclictest17937-21grep00:10:010
166899220,21cyclictest23591-21apt-get19:56:210
166899220,21cyclictest13-21ksoftirqd/021:26:220
166899220,21cyclictest0-21swapper/023:01:220
166899220,21cyclictest0-21swapper/022:06:220
166899220,21cyclictest0-21swapper/020:10:010
166899220,21cyclictest0-21swapper/019:46:210
166899220,21cyclictest0-21swapper/000:21:230
166899220,1cyclictest14157-21apt-get22:56:210
166899220,1cyclictest13-21ksoftirqd/023:45:000
1682992120,1cyclictest24903-21apt-get22:11:223
1682992119,1cyclictest14947-21perf21:50:013
168299211,0cyclictest7704-21apt-get21:36:213
168299210,21cyclictest0-21swapper/322:46:223
168299210,21cyclictest0-21swapper/319:51:223
168299210,21cyclictest0-21swapper/319:46:213
168299210,20cyclictest30659-21turbostat23:30:003
168299210,20cyclictest0-21swapper/322:21:203
168299210,0cyclictest8448-21apt-get20:31:203
168299210,0cyclictest4556-21apt-get22:36:213
1677992121,0cyclictest15305-21apt-get21:51:212
1677992121,0cyclictest0-21swapper/222:26:202
1677992120,0cyclictest8618-21apt-get23:51:222
1677992120,0cyclictest28104-21apt-get21:11:212
167799210,20cyclictest0-21swapper/223:36:202
167799210,20cyclictest0-21swapper/220:26:202
167799210,20cyclictest0-21swapper/219:56:212
167799210,20cyclictest0-21swapper/219:21:202
167799210,0cyclictest9758-21apt-get21:40:022
167799210,0cyclictest0-21swapper/220:46:222
1673992121,0cyclictest0-21swapper/100:25:021
1673992120,0cyclictest0-21swapper/123:01:211
1673992120,0cyclictest0-21swapper/121:16:211
1673992119,1cyclictest3266-21sh20:20:001
1673992119,1cyclictest0-21swapper/119:55:011
167399210,20cyclictest0-21swapper/121:21:211
167399210,20cyclictest0-21swapper/120:31:201
167399210,0cyclictest0-21swapper/119:36:201
167399210,0cyclictest0-21swapper/119:31:211
167399210,0cyclictest0-21swapper/100:06:231
1668992120,1cyclictest0-21swapper/022:01:210
1668992120,1cyclictest0-21swapper/021:36:220
1668992120,0cyclictest25920-21apt-get00:26:250
1668992120,0cyclictest0-21swapper/020:46:210
166899210,21cyclictest0-21swapper/019:41:210
166899210,20cyclictest0-21swapper/023:06:220
166899210,0cyclictest0-21swapper/022:51:200
166899210,0cyclictest0-21swapper/022:36:210
1682992020,0cyclictest0-21swapper/321:40:023
1682992020,0cyclictest0-21swapper/320:56:223
1682992020,0cyclictest0-21swapper/319:16:203
1682992020,0cyclictest0-21swapper/300:36:213
1682992019,1cyclictest456-21apt-get21:21:203
1682992019,0cyclictest0-21swapper/320:11:233
1682992018,1cyclictest8271-21perf23:50:013
1682992018,1cyclictest606-21dbus-daemon23:15:013
168299200,20cyclictest0-21swapper/321:11:213
168299200,0cyclictest0-21swapper/322:31:223
1677992020,0cyclictest9199-21apt-get19:26:202
1677992020,0cyclictest0-21swapper/220:41:212
1677992020,0cyclictest0-21swapper/200:36:212
1677992019,1cyclictest5305-21apt-get21:31:192
1677992019,1cyclictest5305-21apt-get21:31:192
1677992019,1cyclictest23130-21apt-get00:21:232
1677992019,0cyclictest22509-21apt-get22:06:222
1677992019,0cyclictest0-21swapper/221:46:232
1677992018,1cyclictest4385-21apt-get19:16:212
1677992018,1cyclictest3305-21munin-run20:20:002
167799200,20cyclictest0-21swapper/220:01:212
167799200,0cyclictest5464-21apt-get23:45:022
167799200,0cyclictest28312-21apt-get00:31:212
167799200,0cyclictest0-21swapper/219:36:202
167799200,0cyclictest0-21swapper/200:16:222
1673992020,0cyclictest26174-21apt-get23:21:221
1673992020,0cyclictest0-21swapper/123:31:221
1673992020,0cyclictest0-21swapper/121:26:221
1673992020,0cyclictest0-21swapper/121:11:211
1673992019,1cyclictest0-21swapper/122:41:231
1673992018,1cyclictest3027-21sendmail23:40:001
167399200,0cyclictest28622-21apt-get23:26:221
167399200,0cyclictest0-21swapper/121:41:211
167399200,0cyclictest0-21swapper/121:06:231
167399200,0cyclictest0-21swapper/119:26:211
1668992020,0cyclictest6219-21apt-get23:46:200
1668992020,0cyclictest0-21swapper/023:51:220
1668992020,0cyclictest0-21swapper/022:31:220
1668992020,0cyclictest0-21swapper/022:26:200
1668992020,0cyclictest0-21swapper/021:21:210
1668992020,0cyclictest0-21swapper/021:05:020
1668992020,0cyclictest0-21swapper/019:51:220
1668992020,0cyclictest0-21swapper/019:26:210
1668992020,0cyclictest0-21swapper/000:36:210
1668992018,1cyclictest30132-21latency_hist21:15:000
1668992018,1cyclictest23411-21munin-plugin-st23:15:010
1668992018,1cyclictest19763-21munin-run22:00:000
1668992017,2cyclictest8084-21idleruntime-cro20:30:000
1668992016,3cyclictest313-21systemd-journal20:55:010
166899200,0cyclictest0-21swapper/021:16:210
166899200,0cyclictest0-21swapper/020:31:200
166899200,0cyclictest0-21swapper/019:36:200
1682991919,0cyclictest0-21swapper/323:06:233
1682991919,0cyclictest0-21swapper/322:06:223
1682991919,0cyclictest0-21swapper/319:41:213
1682991919,0cyclictest0-21swapper/319:31:203
1682991919,0cyclictest0-21swapper/319:21:203
1682991918,0cyclictest0-21swapper/321:31:193
1682991918,0cyclictest0-21swapper/321:31:193
1682991917,1cyclictest30498-21apt-get21:16:213
168299190,18cyclictest23754-21apt-get23:16:223
1677991919,0cyclictest0-21swapper/223:35:002
1677991919,0cyclictest0-21swapper/223:26:222
1677991919,0cyclictest0-21swapper/222:56:212
1677991919,0cyclictest0-21swapper/222:36:212
1677991919,0cyclictest0-21swapper/221:36:222
1677991919,0cyclictest0-21swapper/221:26:222
1677991919,0cyclictest0-21swapper/221:21:212
1677991919,0cyclictest0-21swapper/221:01:212
167799190,18cyclictest0-21swapper/200:11:212
1673991919,0cyclictest1996-21apt-get19:11:231
1673991919,0cyclictest0-21swapper/119:56:221
1673991919,0cyclictest0-21swapper/100:31:221
1673991919,0cyclictest0-21swapper/100:01:231
1673991918,0cyclictest0-21swapper/121:01:211
1673991918,0cyclictest0-21swapper/119:16:201
1673991917,1cyclictest606-21dbus-daemon20:55:011
166899192,1cyclictest151rcu_preempt22:20:010
1668991919,0cyclictest0-21swapper/023:26:220
1668991919,0cyclictest0-21swapper/020:21:210
1668991919,0cyclictest0-21swapper/020:11:230
1668991919,0cyclictest0-21swapper/019:11:230
1668991919,0cyclictest0-21swapper/000:31:210
1668991918,0cyclictest0-21swapper/021:40:020
1668991916,2cyclictest31818-21munin-plugin-st22:25:010
1682991818,0cyclictest0-21swapper/323:31:223
1682991818,0cyclictest0-21swapper/300:16:233
1682991817,1cyclictest0-21swapper/300:06:223
1677991818,0cyclictest0-21swapper/220:21:212
1677991817,0cyclictest21199-21apt-get19:51:222
1673991818,0cyclictest0-21swapper/123:51:221
1673991818,0cyclictest0-21swapper/123:41:231
1673991818,0cyclictest0-21swapper/119:41:211
1673991817,1cyclictest30704-21apt-get00:36:211
1668991818,0cyclictest20701-21apt-get00:16:220
1668991818,0cyclictest0-21swapper/023:31:220
1668991818,0cyclictest0-21swapper/023:21:220
1668991818,0cyclictest0-21swapper/000:01:230
1668991817,0cyclictest12912-21apt-get21:46:230
1682991717,0cyclictest18304-21apt-get00:11:213
1682991717,0cyclictest0-21swapper/321:01:213
1682991717,0cyclictest0-21swapper/320:51:213
1682991716,0cyclictest0-21swapper/319:26:213
1682991714,2cyclictest14398-21expr22:56:223
1677991717,0cyclictest0-21swapper/222:21:202
1677991715,1cyclictest24882-21apt-get21:05:022
167799170,16cyclictest0-21swapper/222:35:012
1673991716,1cyclictest0-21swapper/122:36:211
1673991716,0cyclictest28455-21apt-get20:06:211
1673991715,1cyclictest13263-21apt-get20:41:211
1673991715,1cyclictest0-21swapper/120:26:261
167399170,16cyclictest6805-21apt-get19:21:201
1668991717,0cyclictest0-21swapper/020:36:190
1668991717,0cyclictest0-21swapper/020:01:210
1668991715,1cyclictest6419-21apt19:20:010
166899170,16cyclictest3033-21sendmail23:40:000
1682991616,0cyclictest15658-21apt-get20:46:223
1682991616,0cyclictest0-21swapper/323:51:223
1682991616,0cyclictest0-21swapper/300:31:223
1682991614,1cyclictest29405-21lockfile-create22:20:013
1682991614,1cyclictest0-21swapper/321:30:013
168299160,15cyclictest18582-21cron23:05:013
1677991616,0cyclictest0-21swapper/223:11:222
1677991616,0cyclictest0-21swapper/222:46:222
1677991615,0cyclictest0-21swapper/222:11:212
1677991614,1cyclictest0-21swapper/223:04:552
1677991614,1cyclictest0-21swapper/220:10:012
1677991613,2cyclictest13773-21cron22:55:002
1673991616,0cyclictest0-21swapper/122:46:231
1673991616,0cyclictest0-21swapper/100:11:211
1673991614,1cyclictest0-21swapper/123:10:011
1682991515,0cyclictest6975-21apt-get22:41:233
1682991515,0cyclictest0-21swapper/321:51:213
1682991515,0cyclictest0-21swapper/320:41:213
1677991513,1cyclictest18966-21apt-get23:06:222
167799150,14cyclictest10504-21munin-plugin-st20:35:012
1673991515,0cyclictest0-21swapper/121:31:191
1673991515,0cyclictest0-21swapper/121:31:191
1673991513,1cyclictest22116-21sed00:16:301
1668991515,0cyclictest0-21swapper/020:16:210
1668991515,0cyclictest0-21swapper/019:31:200
168299140,14cyclictest0-21swapper/323:40:103
168299140,14cyclictest0-21swapper/323:20:593
168299140,14cyclictest0-21swapper/322:00:263
168299140,14cyclictest0-21swapper/321:55:583
168299140,14cyclictest0-21swapper/319:35:583
168299140,0cyclictest0-21swapper/320:05:103
168299140,0cyclictest0-21swapper/320:02:383
1677991414,0cyclictest0-21swapper/223:20:152
1677991414,0cyclictest0-21swapper/220:56:212
1677991414,0cyclictest0-21swapper/219:40:212
167799140,14cyclictest0-21swapper/221:56:082
167799140,14cyclictest0-21swapper/221:15:172
167799140,14cyclictest0-21swapper/219:11:582
167799140,0cyclictest0-21swapper/223:40:032
167799140,0cyclictest0-21swapper/222:00:092
167799140,0cyclictest0-21swapper/220:36:222
167799140,0cyclictest0-21swapper/219:45:352
167799140,0cyclictest0-21swapper/200:00:362
1673991414,0cyclictest0-21swapper/122:50:401
167399140,14cyclictest0-21swapper/122:30:511
167399140,14cyclictest0-21swapper/120:00:531
167399140,0cyclictest0-21swapper/119:45:311
166899140,14cyclictest0-21swapper/023:15:530
166899140,14cyclictest0-21swapper/021:50:320
166899140,0cyclictest0-21swapper/019:20:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional