You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-27 - 20:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Dec 27, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19434992624,2cyclictest0-21swapper/311:15:273
19428992625,1cyclictest19130-21kworker/1:107:30:281
19432992522,2cyclictest21774-21cron07:15:002
19428992523,1cyclictest0-21swapper/109:40:001
19428992519,1cyclictest151rcu_preempt11:20:291
1942899250,24cyclictest8910-21apt-get11:15:021
1942899250,24cyclictest25948-21cron08:30:011
19428992421,2cyclictest606-21dbus-daemon10:45:011
19434992323,0cyclictest0-21swapper/307:10:273
1943499232,20cyclictest0-21swapper/308:35:273
19434992321,1cyclictest18750-21dump-pmu-power08:15:003
1943499230,2cyclictest8597-21basename12:20:003
19432992322,1cyclictest12051-21cron10:15:012
19428992322,0cyclictest17147-21apt-get11:30:021
19434992221,1cyclictest21157-21ls08:20:013
19434992220,1cyclictest21668-21sh10:35:003
1943499220,21cyclictest27648-21awk09:40:003
1943499220,21cyclictest0-21swapper/311:50:013
1943499220,21cyclictest0-21swapper/308:00:273
1943499220,1cyclictest4357-21latency_hist07:45:013
19432992221,1cyclictest30322-21apt-get09:45:272
19432992220,1cyclictest31967-21apt-get10:55:282
1943299221,20cyclictest783-21snmpd08:20:012
1943299220,21cyclictest6004-21idleruntime-cro08:55:002
1943299220,21cyclictest0-21swapper/210:50:272
1943299220,1cyclictest1562-21apt-get07:40:022
19428992220,1cyclictest4136-21apt11:05:011
19428992220,1cyclictest11572-21sendmail08:00:001
1942899220,21cyclictest21967-21apt-get10:35:271
1942899220,21cyclictest14252-21apt-get08:05:271
1942899220,1cyclictest0-21swapper/112:10:271
19426992220,1cyclictest0-21swapper/012:35:010
19426992220,1cyclictest0-21swapper/012:35:010
19426992220,1cyclictest0-21swapper/011:30:000
19426992219,2cyclictest9679-21perf10:10:010
1942699220,21cyclictest3226-21apt-get08:50:020
1942699220,21cyclictest31325-21perf12:00:010
1942699220,21cyclictest2445-21cron09:55:000
1942699220,21cyclictest0-21swapper/011:15:280
1942699220,21cyclictest0-21swapper/009:25:300
1942699220,1cyclictest4424-21apt-get11:05:290
1942699220,1cyclictest1562-21apt-get07:40:010
1942699220,1cyclictest0-21swapper/009:10:260
19434992121,0cyclictest11106-21apt-get09:05:283
19434992121,0cyclictest0-21swapper/309:00:273
19434992121,0cyclictest0-21swapper/307:50:293
19434992120,1cyclictest30027-21dump-pmu-power09:45:003
19434992120,1cyclictest15897-21apt09:15:273
1943499210,20cyclictest18316-21apt-get09:20:273
1943499210,20cyclictest0-21swapper/310:00:283
1943499210,20cyclictest0-21swapper/310:00:283
1943499210,1cyclictest0-21swapper/308:30:273
1943499210,0cyclictest13670-21apt-get12:30:273
1943499210,0cyclictest13670-21apt-get12:30:273
19432992121,0cyclictest0-21swapper/212:00:292
19432992119,1cyclictest4877-21apt10:00:012
19432992119,1cyclictest24191-21cron07:20:002
19432992119,1cyclictest24154-21munin-run11:45:012
1943299210,21cyclictest0-21swapper/208:35:272
1943299210,20cyclictest29218-21apt-get11:55:282
1943299210,20cyclictest21464-21apt-get08:20:252
1943299210,20cyclictest0-21swapper/212:05:302
1943299210,20cyclictest0-21swapper/209:20:272
1943299210,1cyclictest0-21swapper/211:15:022
1943299210,1cyclictest0-21swapper/210:40:282
1943299210,1cyclictest0-21swapper/208:10:272
1943299210,0cyclictest0-21swapper/210:20:282
19428992119,1cyclictest0-21swapper/107:25:001
1942899210,21cyclictest0-21swapper/110:55:021
1942899210,20cyclictest26261-21apt-get08:30:271
1942899210,20cyclictest0-21swapper/108:10:271
1942899210,20cyclictest0-21swapper/107:40:011
1942899210,1cyclictest0-21swapper/112:30:271
1942899210,1cyclictest0-21swapper/112:30:271
1942899210,0cyclictest22032-21apt-get11:40:271
1942899210,0cyclictest0-21swapper/111:05:301
1942899210,0cyclictest0-21swapper/110:50:271
1942899210,0cyclictest0-21swapper/109:45:271
1942899210,0cyclictest0-21swapper/108:50:271
19426992119,1cyclictest30049-21munin-run09:45:000
19426992119,1cyclictest25209-21dump-pmu-power09:35:000
19426992119,1cyclictest14135-21perf11:25:000
19426992119,1cyclictest0-21swapper/009:25:010
1942699210,21cyclictest0-21swapper/008:05:270
1942699210,20cyclictest0-21swapper/012:00:280
1942699210,20cyclictest0-21swapper/011:10:290
1942699210,20cyclictest0-21swapper/011:00:300
1942699210,20cyclictest0-21swapper/010:55:280
1942699210,0cyclictest0-21swapper/012:20:280
1942699210,0cyclictest0-21swapper/011:50:290
19434992020,0cyclictest0-21swapper/312:10:003
19434992020,0cyclictest0-21swapper/309:10:263
19434992020,0cyclictest0-21swapper/308:40:293
19434992019,1cyclictest0-21swapper/310:40:283
19434992019,1cyclictest0-21swapper/307:55:283
19434992018,1cyclictest0-21swapper/309:50:003
1943499200,0cyclictest0-21swapper/311:50:293
1943499200,0cyclictest0-21swapper/311:30:023
1943499200,0cyclictest0-21swapper/311:20:283
1943499200,0cyclictest0-21swapper/310:55:283
1943499200,0cyclictest0-21swapper/310:05:313
1943499200,0cyclictest0-21swapper/309:30:273
1943499200,0cyclictest0-21swapper/307:25:273
1943499200,0cyclictest0-21swapper/307:15:263
19432992020,0cyclictest11276-21apt-get12:25:292
19432992020,0cyclictest0-21swapper/211:00:302
19432992020,0cyclictest0-21swapper/210:30:272
19432992020,0cyclictest0-21swapper/209:50:292
19432992020,0cyclictest0-21swapper/207:25:272
19432992019,1cyclictest14419-21apt-get11:25:272
19432992019,0cyclictest5173-21apt-get10:00:282
19432992019,0cyclictest5173-21apt-get10:00:282
19432992018,1cyclictest8906-21cron11:15:002
1943299200,0cyclictest0-21swapper/211:35:272
1943299200,0cyclictest0-21swapper/211:05:302
19428992020,0cyclictest6297-21apt-get08:55:271
19428992020,0cyclictest20721-21apt-get09:25:301
19428992020,0cyclictest19039-21apt-get08:15:281
19428992020,0cyclictest11858-21apt-get08:00:271
19428992020,0cyclictest0-21swapper/112:35:311
19428992020,0cyclictest0-21swapper/111:50:291
19428992020,0cyclictest0-21swapper/110:45:291
19428992020,0cyclictest0-21swapper/107:45:281
19428992020,0cyclictest0-21swapper/107:10:271
19428992019,1cyclictest27930-21apt-get09:40:281
19428992018,1cyclictest22-21ksoftirqd/108:35:271
19428992018,1cyclictest16539-21munin-run11:30:001
1942899200,20cyclictest0-21swapper/112:25:291
1942899200,20cyclictest0-21swapper/109:05:281
1942899200,0cyclictest0-21swapper/109:50:291
19426992020,0cyclictest0-21swapper/010:35:270
19426992020,0cyclictest0-21swapper/010:00:280
19426992020,0cyclictest0-21swapper/010:00:280
19426992020,0cyclictest0-21swapper/009:55:280
19426992020,0cyclictest0-21swapper/008:35:270
19426992020,0cyclictest0-21swapper/008:20:250
19426992020,0cyclictest0-21swapper/007:10:270
19426992019,1cyclictest0-21swapper/008:25:270
1942699200,20cyclictest0-21swapper/008:55:270
1942699200,0cyclictest0-21swapper/010:10:280
1942699200,0cyclictest0-21swapper/007:50:280
1943499192,16cyclictest0-21swapper/311:10:353
19434991919,0cyclictest2757-21apt-get09:55:283
19434991919,0cyclictest0-21swapper/311:40:283
19434991919,0cyclictest0-21swapper/311:05:303
19434991919,0cyclictest0-21swapper/307:45:283
19434991919,0cyclictest0-21swapper/307:20:263
19434991918,1cyclictest0-21swapper/308:25:013
19434991918,0cyclictest0-21swapper/312:20:283
19434991917,2cyclictest0-21swapper/310:15:283
19432991919,0cyclictest26824-21apt-get11:50:292
19432991919,0cyclictest0-21swapper/207:50:282
19432991918,1cyclictest6461-21apt-get12:15:272
19432991918,1cyclictest31698-21apt-get07:35:272
19432991918,1cyclictest0-21swapper/209:40:292
19432991918,0cyclictest13503-21apt-get09:10:272
19432991917,1cyclictest4653-21apt-get07:45:272
19432991917,1cyclictest23859-21apt-get08:25:262
19432991917,1cyclictest13955-21cron08:05:002
19428991919,0cyclictest1670-21apt-get12:05:301
19428991918,1cyclictest9961-21apt-get10:10:281
19428991918,1cyclictest4358-21awk07:45:011
19426991919,0cyclictest0-21swapper/011:45:280
19426991919,0cyclictest0-21swapper/011:30:310
19426991919,0cyclictest0-21swapper/008:15:280
19426991919,0cyclictest0-21swapper/007:15:270
19426991918,1cyclictest0-21swapper/009:00:280
19426991918,0cyclictest24384-21apt-get10:40:280
19434991818,0cyclictest14774-21apt-get10:20:283
19434991818,0cyclictest0-21swapper/312:00:283
19434991818,0cyclictest0-21swapper/308:55:273
19434991818,0cyclictest0-21swapper/307:35:273
19434991817,1cyclictest0-21swapper/311:55:273
19434991817,0cyclictest1098-21apt-get08:45:273
19434991816,2cyclictest16127-21grep11:25:363
19434991815,2cyclictest4129-21latency_hist11:05:013
19432991818,0cyclictest0-21swapper/211:30:312
19432991818,0cyclictest0-21swapper/210:45:292
19432991818,0cyclictest0-21swapper/209:35:272
19432991818,0cyclictest0-21swapper/209:25:302
19432991818,0cyclictest0-21swapper/209:05:282
19428991818,0cyclictest0-21swapper/110:30:271
19428991818,0cyclictest0-21swapper/110:15:281
19428991818,0cyclictest0-21swapper/110:00:281
19428991818,0cyclictest0-21swapper/110:00:281
19428991816,1cyclictest21727-21dump-pmu-power11:40:011
19426991818,0cyclictest29175-21apt-get10:50:270
19426991816,1cyclictest4067-21apt-get12:10:260
19434991716,1cyclictest21967-21apt-get10:35:283
19434991716,1cyclictest19285-21perf10:30:003
19434991715,2cyclictest5110-21ntp_states12:10:323
19434991715,2cyclictest0-21swapper/308:50:273
19434991715,2cyclictest0-21swapper/307:30:363
19434991714,2cyclictest0-21swapper/308:05:273
19434991712,4cyclictest0-21swapper/310:10:313
19432991717,0cyclictest0-21swapper/211:20:292
19432991717,0cyclictest0-21swapper/208:45:272
19432991717,0cyclictest0-21swapper/208:05:272
19432991715,1cyclictest18202-21sendmail12:40:012
19428991717,0cyclictest0-21swapper/110:05:311
19428991716,1cyclictest26549-21munin-run11:50:011
19428991716,1cyclictest22-21ksoftirqd/108:50:011
19428991715,2cyclictest22-21ksoftirqd/107:55:011
19428991715,1cyclictest8933-21perf11:15:001
19428991715,1cyclictest622-21gdbus10:00:011
19428991715,1cyclictest0-21swapper/110:30:001
19426991717,0cyclictest29304-21apt-get07:30:280
19426991717,0cyclictest12354-21apt-get10:15:280
19426991717,0cyclictest0-21swapper/012:25:290
19426991717,0cyclictest0-21swapper/007:55:280
19426991717,0cyclictest0-21swapper/007:25:270
19426991717,0cyclictest0-21swapper/007:20:260
19426991716,1cyclictest21693-21munin-run10:35:000
19426991716,1cyclictest0-21swapper/008:45:280
19426991715,1cyclictest6772-21idleruntime-cro07:50:010
19434991615,1cyclictest29389-21cstates10:50:283
19434991615,1cyclictest0-21swapper/309:55:013
19434991614,1cyclictest18171-21latency_hist12:40:013
19432991616,0cyclictest0-21swapper/212:30:272
19432991616,0cyclictest0-21swapper/212:30:272
19432991616,0cyclictest0-21swapper/209:00:272
19432991616,0cyclictest0-21swapper/208:30:272
19432991616,0cyclictest0-21swapper/207:30:282
19432991615,0cyclictest0-21swapper/208:40:292
19432991614,1cyclictest25201-21sh09:35:002
1943299160,15cyclictest313-21systemd-journal10:26:122
1943299160,10cyclictest611-21irqbalance07:22:202
19428991616,0cyclictest8712-21apt-get09:00:271
19428991616,0cyclictest0-21swapper/109:10:271
19426991616,0cyclictest31074-21apt-get08:40:290
19426991616,0cyclictest0-21swapper/011:35:270
19426991616,0cyclictest0-21swapper/009:10:010
19434991513,2cyclictest25986-21awk08:30:013
19432991515,0cyclictest0-21swapper/210:35:272
19428991515,0cyclictest0-21swapper/111:55:281
19428991515,0cyclictest0-21swapper/109:20:271
1942899150,14cyclictest16831-21missed_timers09:15:321
1942699153,11cyclictest62550irq/127-eno109:45:270
19426991515,0cyclictest0-21swapper/012:35:310
19426991515,0cyclictest0-21swapper/011:40:280
19426991515,0cyclictest0-21swapper/007:40:260
19426991513,1cyclictest17902-21modprobe10:25:310
19426991511,4cyclictest0-21swapper/009:35:550
1942699150,1cyclictest622-21gdbus10:25:000
1942699150,14cyclictest0-21swapper/010:45:360
19434991414,0cyclictest0-21swapper/311:35:133
1943499140,14cyclictest0-21swapper/310:45:083
1943499140,14cyclictest0-21swapper/309:25:033
1943499140,0cyclictest0-21swapper/312:25:113
19432991414,0cyclictest0-21swapper/212:10:242
19432991414,0cyclictest0-21swapper/207:55:282
19432991413,1cyclictest0-21swapper/208:55:292
1943299140,14cyclictest0-21swapper/210:15:272
1943299140,0cyclictest0-21swapper/212:21:302
1943299140,0cyclictest0-21swapper/211:45:042
1943299140,0cyclictest0-21swapper/210:05:022
1943299140,0cyclictest0-21swapper/209:15:022
19428991414,0cyclictest26907-21apt-get07:25:271
19428991414,0cyclictest0-21swapper/112:21:081
19428991414,0cyclictest0-21swapper/108:20:261
19428991414,0cyclictest0-21swapper/107:15:041
1942899140,14cyclictest0-21swapper/112:00:101
1942899140,0cyclictest0-21swapper/110:21:431
19426991414,0cyclictest0-21swapper/012:05:020
19426991414,0cyclictest0-21swapper/008:31:020
19426991414,0cyclictest0-21swapper/008:00:130
1942699140,14cyclictest0-21swapper/008:10:410
1942699140,0cyclictest0-21swapper/012:15:130
1942699140,0cyclictest0-21swapper/009:18:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional