You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 11:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Wed Dec 17, 2025 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1625399290,1cyclictest313-21systemd-journal00:15:010
1625399270,23cyclictest29647-21apt-get19:39:320
1626799240,23cyclictest20716-21munin-plugin-st20:25:003
1626599240,23cyclictest0-21swapper/220:59:332
1626599240,0cyclictest0-21swapper/220:24:362
1626399240,23cyclictest15087-21idleruntime-cro21:20:001
16267992320,2cyclictest11059-21grep20:05:013
1626799230,22cyclictest0-21swapper/321:05:033
16265992321,1cyclictest28422-21munin-run22:55:002
1626599231,1cyclictest0-21swapper/221:35:012
1626599230,2cyclictest15826-21dump-pmu-power20:15:012
1626599230,22cyclictest30184-21ls00:05:012
1626599230,22cyclictest0-21swapper/220:10:012
16263992322,1cyclictest0-21swapper/122:49:341
1626399230,22cyclictest211rcuc/122:25:011
1626399230,22cyclictest18299-21apt20:20:011
1626399230,22cyclictest11419-21apt-get23:29:341
1626399230,22cyclictest0-21swapper/123:49:341
1625399230,22cyclictest4966-21apt-get00:20:010
16267992222,0cyclictest0-21swapper/323:59:323
16267992220,1cyclictest4338-21sh22:05:003
16267992220,1cyclictest3812-21cron19:50:003
16267992220,1cyclictest32564-21apt-get21:59:333
16267992220,1cyclictest19973-21awk21:30:013
16267992220,1cyclictest12693-21munin-run21:15:003
16267992220,1cyclictest0-21swapper/323:10:023
16267992220,1cyclictest0-21swapper/319:54:353
1626799220,21cyclictest24755-21sendmail-msp21:40:013
1626799220,21cyclictest0-21swapper/322:54:333
1626799220,21cyclictest0-21swapper/321:55:013
16265992219,2cyclictest23109-21munin-run20:30:002
1626599220,21cyclictest30827-21cron23:00:002
1626599220,21cyclictest2049-21apt-get19:49:332
1626599220,21cyclictest0-21swapper/221:59:332
1626599220,21cyclictest0-21swapper/220:44:342
1626599220,1cyclictest0-21swapper/221:29:332
16263992221,1cyclictest26626-21sh19:30:001
16263992220,1cyclictest16366-21munin-run22:30:011
16263992220,1cyclictest12684-21dump-pmu-power21:15:001
16263992220,1cyclictest0-21swapper/121:50:011
1626399220,21cyclictest890-21apt-get20:54:341
1626399220,21cyclictest8430-21perf23:20:011
1626399220,21cyclictest26106-21apt-get20:39:341
1626399220,21cyclictest0-21swapper/123:39:341
1626399220,1cyclictest28525-21apt-get20:44:341
1626399220,1cyclictest10803-21latency_hist23:25:011
16253992222,0cyclictest18691-21apt-get23:44:350
16253992222,0cyclictest0-21swapper/021:09:340
1625399220,21cyclictest6842-21apt-get19:59:350
1625399220,21cyclictest5999-21perf23:15:010
1625399220,21cyclictest5057-21cron21:00:010
1625399220,21cyclictest24855-21apt-get19:29:330
1625399220,21cyclictest23482-21apt-get23:54:340
1625399220,21cyclictest18923-21apt-get20:24:370
1625399220,21cyclictest15611-21perf23:35:010
1625399220,21cyclictest0-21swapper/020:09:330
1625399220,21cyclictest0-21swapper/019:34:340
1625399220,1cyclictest18169-21apt-get21:29:330
1625399220,1cyclictest0-21swapper/023:05:000
16267992121,0cyclictest21836-21apt-get22:44:353
16267992121,0cyclictest0-21swapper/300:04:343
16267992120,0cyclictest0-21swapper/320:54:343
16267992119,1cyclictest5715-21apt-get21:04:333
1626799210,21cyclictest0-21swapper/320:19:323
1626799210,20cyclictest5979-21apt-get00:24:333
1626799210,20cyclictest0-21swapper/323:39:343
1626799210,20cyclictest0-21swapper/322:49:343
1626799210,20cyclictest0-21swapper/322:29:343
1626799210,20cyclictest0-21swapper/319:24:343
1626799210,0cyclictest0-21swapper/323:49:343
1626799210,0cyclictest0-21swapper/321:49:353
16265992121,0cyclictest0-21swapper/223:19:342
16265992121,0cyclictest0-21swapper/220:04:342
16265992120,1cyclictest16162-21apt-get19:10:022
16265992120,1cyclictest13299-21apt-get21:19:352
16265992120,1cyclictest0-21swapper/221:54:332
16265992120,1cyclictest0-21swapper/220:39:342
16265992119,1cyclictest24239-21cron19:25:012
16265992119,1cyclictest0-21swapper/219:55:012
1626599210,21cyclictest0-21swapper/200:30:002
1626599210,20cyclictest31439-21apt-get23:04:332
1626599210,1cyclictest16215-21apt-get23:39:342
16263992121,0cyclictest9258-21apt-get20:04:341
16263992120,0cyclictest4967-21apt-get22:09:351
16263992119,1cyclictest24783-21awk21:40:011
16263992118,2cyclictest14968-21perf00:40:001
1626399210,21cyclictest0-21swapper/122:44:351
1626399210,20cyclictest0-21swapper/120:59:321
1626399210,20cyclictest0-21swapper/100:14:351
1626399210,0cyclictest3169-21apt-get00:19:341
1626399210,0cyclictest0-21swapper/123:59:321
16253992121,0cyclictest0-21swapper/022:49:340
16253992121,0cyclictest0-21swapper/022:29:340
16253992120,0cyclictest16215-21apt-get23:39:340
16253992119,1cyclictest25489-21latency_hist20:35:000
1625399210,21cyclictest0-21swapper/021:14:350
1625399210,21cyclictest0-21swapper/000:04:340
1625399210,20cyclictest0-21swapper/022:19:340
1625399210,20cyclictest0-21swapper/020:04:340
1625399210,0cyclictest0-21swapper/020:54:340
1625399210,0cyclictest0-21swapper/020:19:320
16267992020,0cyclictest0-21swapper/323:55:003
16267992020,0cyclictest0-21swapper/323:34:333
16267992020,0cyclictest0-21swapper/320:59:333
16267992020,0cyclictest0-21swapper/319:34:333
16267992020,0cyclictest0-21swapper/319:15:023
16267992020,0cyclictest0-21swapper/300:19:343
16267992019,0cyclictest11650-21apt-get20:09:333
1626799200,0cyclictest0-21swapper/322:14:343
1626799200,0cyclictest0-21swapper/320:29:313
16265992020,0cyclictest30778-21apt-get00:09:352
16265992020,0cyclictest0-21swapper/223:54:342
16265992020,0cyclictest0-21swapper/223:49:342
16265992020,0cyclictest0-21swapper/223:44:352
16265992020,0cyclictest0-21swapper/223:29:342
16265992020,0cyclictest0-21swapper/222:49:342
16265992020,0cyclictest0-21swapper/222:24:332
16265992020,0cyclictest0-21swapper/219:59:352
16265992019,1cyclictest0-21swapper/223:09:342
16265992019,0cyclictest0-21swapper/220:49:322
16265992017,2cyclictest313-21systemd-journal23:35:012
1626599200,20cyclictest0-21swapper/222:34:332
1626599200,0cyclictest0-21swapper/221:39:342
1626599200,0cyclictest0-21swapper/219:39:322
16263992020,0cyclictest0-21swapper/121:04:341
16263992020,0cyclictest0-21swapper/120:29:321
16263992020,0cyclictest0-21swapper/100:29:351
1626399200,20cyclictest0-21swapper/120:24:371
1626399200,20cyclictest0-21swapper/119:10:021
1626399200,20cyclictest0-21swapper/100:07:391
1626399200,0cyclictest23713-21apt-get20:34:341
1626399200,0cyclictest0-21swapper/122:19:341
1626399200,0cyclictest0-21swapper/100:04:341
16253992020,0cyclictest30916-21apt-get20:49:320
16253992020,0cyclictest22962-21apt-get21:39:350
16253992020,0cyclictest0-21swapper/023:29:340
16253992020,0cyclictest0-21swapper/022:44:350
16253992020,0cyclictest0-21swapper/021:59:330
16253992020,0cyclictest0-21swapper/021:49:350
16253992020,0cyclictest0-21swapper/021:44:340
16253992020,0cyclictest0-21swapper/021:04:340
16253992020,0cyclictest0-21swapper/019:44:310
1625399200,0cyclictest0-21swapper/023:49:340
1625399200,0cyclictest0-21swapper/022:39:330
1625399200,0cyclictest0-21swapper/022:09:340
1625399200,0cyclictest0-21swapper/000:24:330
16267991919,0cyclictest0-21swapper/323:44:353
16267991919,0cyclictest0-21swapper/323:19:343
16267991919,0cyclictest0-21swapper/322:39:343
16267991919,0cyclictest0-21swapper/321:44:343
16267991919,0cyclictest0-21swapper/321:24:343
16267991919,0cyclictest0-21swapper/320:44:343
16267991919,0cyclictest0-21swapper/320:39:343
16267991919,0cyclictest0-21swapper/319:39:323
16267991917,1cyclictest0-21swapper/300:39:333
16265991919,0cyclictest0-21swapper/223:24:342
16265991917,1cyclictest29042-21idleruntime-cro19:35:002
16263991919,0cyclictest16972-21apt-get22:34:341
16263991919,0cyclictest0-21swapper/119:54:341
16263991919,0cyclictest0-21swapper/119:19:351
16263991919,0cyclictest0-21swapper/100:34:361
16253991919,0cyclictest20569-21apt-get21:34:350
16253991919,0cyclictest0-21swapper/023:09:340
16253991919,0cyclictest0-21swapper/022:04:350
16253991919,0cyclictest0-21swapper/000:34:360
16253991918,1cyclictest32572-21perf00:10:010
16253991918,1cyclictest25874-21apt-get23:59:320
16267991818,0cyclictest0-21swapper/321:19:353
16267991818,0cyclictest0-21swapper/320:14:343
16267991816,1cyclictest19044-21apt-get19:15:013
16267991816,1cyclictest10163-21latency_hist00:30:003
16265991818,0cyclictest8505-21apt-get21:09:342
16265991818,0cyclictest0-21swapper/200:19:342
16265991816,1cyclictest619-21polkitd21:25:002
16265991816,1cyclictest313-21systemd-journal20:55:012
1626599180,17cyclictest606-21dbus-daemon22:15:002
16263991818,0cyclictest15716-21apt-get21:24:351
16263991818,0cyclictest0-21swapper/123:54:341
16263991818,0cyclictest0-21swapper/122:14:341
16263991818,0cyclictest0-21swapper/121:54:331
16263991818,0cyclictest0-21swapper/121:29:331
16263991818,0cyclictest0-21swapper/119:34:341
16263991815,1cyclictest313-21systemd-journal00:25:011
16253991818,0cyclictest6610-21apt-get23:19:330
16253991818,0cyclictest30171-21apt-get21:54:330
16253991818,0cyclictest20046-21apt-get19:19:350
16253991818,0cyclictest0-21swapper/022:59:340
16253991818,0cyclictest0-21swapper/022:34:330
16253991818,0cyclictest0-21swapper/021:19:350
16253991818,0cyclictest0-21swapper/019:10:020
16253991818,0cyclictest0-21swapper/000:29:350
16267991717,0cyclictest0-21swapper/323:09:343
16267991717,0cyclictest0-21swapper/322:59:343
16267991717,0cyclictest0-21swapper/320:34:343
16267991717,0cyclictest0-21swapper/319:29:343
16267991715,1cyclictest13217-21munin-run23:30:013
16267991715,1cyclictest12564-21munin-run00:35:003
16265991717,0cyclictest0-21swapper/222:09:352
16265991717,0cyclictest0-21swapper/200:15:012
16263991717,0cyclictest15842-21perf20:15:011
16263991717,0cyclictest0-21swapper/123:34:331
16263991717,0cyclictest0-21swapper/121:59:331
16263991717,0cyclictest0-21swapper/120:09:331
16263991717,0cyclictest0-21swapper/119:39:321
16263991716,0cyclictest25380-21apt-get21:44:331
16253991717,0cyclictest0-21swapper/021:24:350
16267991616,0cyclictest0-21swapper/322:34:343
16267991616,0cyclictest0-21swapper/322:24:343
16267991616,0cyclictest0-21swapper/321:34:353
16267991616,0cyclictest0-21swapper/319:59:353
16267991616,0cyclictest0-21swapper/300:09:343
16265991616,0cyclictest2574-21apt-get22:04:352
16265991616,0cyclictest0-21swapper/222:44:352
16265991616,0cyclictest0-21swapper/221:49:352
16265991616,0cyclictest0-21swapper/219:15:022
16265991615,1cyclictest13160-21apt-get00:39:332
16265991614,1cyclictest0-21swapper/219:45:002
1626599160,15cyclictest25510-21munin-run20:35:002
1626599160,15cyclictest0-21swapper/222:29:332
16263991616,0cyclictest0-21swapper/123:04:331
16263991616,0cyclictest0-21swapper/121:34:341
16253991616,0cyclictest0-21swapper/019:54:350
16253991616,0cyclictest0-21swapper/000:39:330
16253991614,1cyclictest7361-21apt-get22:14:340
16253991614,1cyclictest26627-21apt-get22:54:330
1625399160,0cyclictest0-21swapper/023:24:380
16267991515,0cyclictest151rcu_preempt19:44:313
16265991515,0cyclictest0-21swapper/223:59:322
16265991515,0cyclictest0-21swapper/219:29:332
16263991515,0cyclictest0-21swapper/122:59:341
16263991515,0cyclictest0-21swapper/119:59:351
16253991515,0cyclictest0-21swapper/020:44:340
1625399150,1cyclictest24264-21perf19:25:010
1625399150,14cyclictest0-21swapper/020:10:530
16267991414,0cyclictest0-21swapper/322:05:393
1626799140,14cyclictest0-21swapper/322:15:113
1626799140,14cyclictest0-21swapper/320:45:543
1626799140,0cyclictest0-21swapper/323:21:293
1626799140,0cyclictest0-21swapper/323:01:043
1626799140,0cyclictest0-21swapper/300:10:193
16265991414,0cyclictest0-21swapper/222:35:012
16265991414,0cyclictest0-21swapper/222:15:022
16265991414,0cyclictest0-21swapper/221:00:142
1626599140,14cyclictest0-21swapper/223:12:512
1626599140,14cyclictest0-21swapper/221:40:132
1626599140,14cyclictest0-21swapper/221:11:002
1626599140,14cyclictest0-21swapper/220:16:062
1626599140,14cyclictest0-21swapper/200:30:142
1626599140,0cyclictest0-21swapper/200:21:192
16263991414,0cyclictest0-21swapper/123:10:181
16263991414,0cyclictest0-21swapper/121:05:141
16263991414,0cyclictest0-21swapper/119:45:071
1626399140,14cyclictest0-21swapper/122:50:181
1626399140,14cyclictest0-21swapper/122:00:221
1626399140,14cyclictest0-21swapper/119:21:321
1626399140,0cyclictest0-21swapper/123:40:411
1626399140,0cyclictest0-21swapper/120:45:151
16253991414,0cyclictest0-21swapper/019:45:220
1625399140,14cyclictest0-21swapper/022:20:130
1625399140,14cyclictest0-21swapper/020:36:130
1625399140,0cyclictest0-21swapper/020:25:380
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional