You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 00:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Thu Dec 25, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9838992623,2cyclictest606-21dbus-daemon08:40:003
9832992522,3cyclictest0-21swapper/109:30:161
983299250,20cyclictest23875-21apt-get08:45:161
983899240,23cyclictest0-21swapper/310:35:163
9836992422,1cyclictest28378-21cron08:55:012
983699240,23cyclictest19629-21apt-get11:55:172
983699240,23cyclictest13238-21ls09:30:002
9831992421,2cyclictest2175-21idleruntime-cro11:20:010
983199240,23cyclictest4479-21apt-get12:30:170
9838992320,2cyclictest28040-21idleruntime-cro10:00:013
983899230,22cyclictest23226-21cron09:50:013
983899230,22cyclictest0-21swapper/307:35:163
983699230,22cyclictest27327-21perf11:05:002
9832992323,0cyclictest0-21swapper/110:05:021
9832992320,2cyclictest22184-21latency_hist07:35:011
983299230,2cyclictest21748-21dump-pmu-power12:00:001
983299230,22cyclictest24664-21sendmail07:40:011
983199230,22cyclictest6934-21sh11:30:010
983199230,22cyclictest6934-21sh11:30:010
9838992220,1cyclictest30508-21cat10:05:013
983899220,21cyclictest2030-21apt-get12:25:023
983899220,21cyclictest14226-21apt-get08:25:173
983899220,21cyclictest0-21swapper/309:35:023
983899220,1cyclictest26269-21apt-get08:50:153
983899220,1cyclictest10449-21latency_hist09:25:003
983899220,1cyclictest0-21swapper/311:35:183
9836992220,1cyclictest6949-21perf11:30:012
9836992220,1cyclictest6949-21perf11:30:012
9836992220,1cyclictest606-21dbus-daemon07:30:012
9836992220,1cyclictest21133-21apt-get09:45:172
9836992220,1cyclictest18766-21cron08:35:002
983699220,21cyclictest27586-21apt-get11:05:172
983699220,21cyclictest0-21swapper/209:30:162
983699220,1cyclictest6614-21perf12:35:012
983699220,1cyclictest28338-21apt-get10:00:162
983699220,1cyclictest2167-21perf11:20:012
9832992216,6cyclictest17233-21apt-get11:50:161
983299220,21cyclictest28406-21munin-run08:55:011
983299220,21cyclictest23529-21apt-get09:50:181
983299220,21cyclictest2050-21/usr/sbin/munin12:25:171
983299220,21cyclictest0-21swapper/111:25:151
983299220,21cyclictest0-21swapper/111:25:151
983299220,0cyclictest0-21swapper/111:30:021
9831992222,0cyclictest0-21swapper/011:55:170
983199220,21cyclictest0-21swapper/009:10:170
9838992120,1cyclictest6943-21apt-get11:30:023
9838992120,0cyclictest0-21swapper/311:00:183
9838992119,1cyclictest18740-21apt-get09:40:163
9838992119,1cyclictest0-21swapper/308:05:193
9838992118,2cyclictest8113-21idleruntime-cro10:25:003
9838992118,2cyclictest13930-21idleruntime-cro08:25:013
983899210,20cyclictest0-21swapper/312:10:193
983899210,20cyclictest0-21swapper/312:05:183
983899210,20cyclictest0-21swapper/308:35:003
983899210,20cyclictest0-21swapper/307:40:153
983899210,1cyclictest0-21swapper/309:10:173
983899210,0cyclictest0-21swapper/310:50:173
983899210,0cyclictest0-21swapper/310:40:183
983899210,0cyclictest0-21swapper/310:25:163
983899210,0cyclictest0-21swapper/309:00:163
983899210,0cyclictest0-21swapper/309:00:163
9836992121,0cyclictest0-21swapper/209:50:182
9836992120,1cyclictest26836-21apt-get12:10:182
9836992120,1cyclictest0-21swapper/210:50:172
9836992120,1cyclictest0-21swapper/209:35:172
9836992120,0cyclictest0-21swapper/212:25:022
9836992119,1cyclictest0-21swapper/210:40:172
983699210,21cyclictest0-21swapper/208:10:152
983699210,21cyclictest0-21swapper/207:20:022
983699210,20cyclictest0-21swapper/209:20:172
983699210,20cyclictest0-21swapper/209:10:172
983699210,1cyclictest16398-21perf08:30:012
983699210,1cyclictest0-21swapper/210:10:162
983699210,0cyclictest5915-21apt-get09:15:182
983699210,0cyclictest0-21swapper/209:40:172
9832992121,0cyclictest23270-21perf09:50:011
9832992120,1cyclictest15593-21apt-get10:40:171
9832992120,0cyclictest25193-21apt-get11:00:181
9832992119,2cyclictest3519-21apt-get09:10:171
9832992119,1cyclictest9101-21cron08:15:011
983299210,21cyclictest22045-21apt-get12:00:171
983299210,21cyclictest0-21swapper/109:55:181
983299210,20cyclictest3563-21apt-get10:15:191
983299210,0cyclictest4620-21apt-get08:05:191
983299210,0cyclictest24440-21apt-get12:05:181
983299210,0cyclictest0-21swapper/111:10:161
9831992121,0cyclictest0-21swapper/012:15:190
9831992121,0cyclictest0-21swapper/011:20:170
9831992119,1cyclictest28377-21sh08:55:010
9831992119,1cyclictest27320-21munin-run11:05:000
983199210,21cyclictest0-21swapper/009:35:020
983199210,21cyclictest0-21swapper/007:20:160
983199210,20cyclictest29821-21apt-get07:50:150
983199210,20cyclictest20380-21apt-get10:50:170
983199210,20cyclictest0-21swapper/008:10:160
983199210,20cyclictest0-21swapper/007:30:170
983199210,1cyclictest12155-21wc11:40:010
983199210,0cyclictest24977-21apt-get07:40:140
983199210,0cyclictest0-21swapper/008:30:170
9838992020,0cyclictest0-21swapper/312:00:173
9838992020,0cyclictest0-21swapper/311:45:183
9838992020,0cyclictest0-21swapper/308:00:173
9838992020,0cyclictest0-21swapper/307:10:163
9838992019,1cyclictest29284-21apt-get12:15:193
9838992019,1cyclictest0-21swapper/312:20:173
9838992019,1cyclictest0-21swapper/307:20:023
9838992018,1cyclictest13527-21apt-get09:30:173
9838992018,1cyclictest13264-21awk09:30:003
9838992018,1cyclictest0-21swapper/312:00:003
9838992017,2cyclictest8014-21sh09:20:013
983899200,20cyclictest0-21swapper/312:31:333
983899200,20cyclictest0-21swapper/311:25:153
983899200,20cyclictest0-21swapper/311:25:153
983899200,19cyclictest14653-21latency_hist07:20:013
983899200,0cyclictest0-21swapper/311:40:173
983899200,0cyclictest0-21swapper/310:10:153
983899200,0cyclictest0-21swapper/307:30:173
9836992020,0cyclictest22775-21apt-get10:55:172
9836992020,0cyclictest0-21swapper/208:45:162
9836992020,0cyclictest0-21swapper/208:40:172
9836992020,0cyclictest0-21swapper/208:20:152
9836992020,0cyclictest0-21swapper/208:15:152
9836992019,1cyclictest14840-21apt-get11:45:182
9836992019,1cyclictest0-21swapper/210:20:162
983699200,0cyclictest0-21swapper/211:35:182
9832992020,0cyclictest16671-21apt-get08:30:171
9832992020,0cyclictest12442-21apt-get11:40:161
9832992020,0cyclictest0-21swapper/111:45:181
9832992020,0cyclictest0-21swapper/111:05:171
9832992020,0cyclictest0-21swapper/110:30:171
9832992020,0cyclictest0-21swapper/110:10:161
9832992020,0cyclictest0-21swapper/109:35:021
9832992020,0cyclictest0-21swapper/108:35:171
9832992020,0cyclictest0-21swapper/108:26:341
9832992019,1cyclictest22508-21perf10:55:011
983299200,20cyclictest0-21swapper/110:45:181
9831992020,0cyclictest17664-21apt-get07:25:150
9831992020,0cyclictest12460-21apt-get07:15:190
9831992020,0cyclictest1154-21apt-get10:10:160
9831992020,0cyclictest0-21swapper/012:20:170
9831992020,0cyclictest0-21swapper/011:30:020
9831992020,0cyclictest0-21swapper/011:10:150
9831992020,0cyclictest0-21swapper/010:00:160
9831992020,0cyclictest0-21swapper/009:50:190
9831992020,0cyclictest0-21swapper/009:30:160
9831992020,0cyclictest0-21swapper/009:15:180
9831992020,0cyclictest0-21swapper/008:35:170
9831992020,0cyclictest0-21swapper/008:05:190
9831992019,1cyclictest13234-21cron09:30:000
9831992018,1cyclictest11565-21idleruntime-cro08:20:010
983199200,20cyclictest0-21swapper/010:25:160
983199200,0cyclictest10065-21apt-get07:10:160
983199200,0cyclictest0-21swapper/010:15:190
983199200,0cyclictest0-21swapper/009:05:170
9838991919,0cyclictest0-21swapper/311:20:163
9838991919,0cyclictest0-21swapper/310:55:173
9838991919,0cyclictest0-21swapper/310:05:193
9838991917,1cyclictest5705-21perf10:20:003
9838991917,1cyclictest10777-21apt-get10:30:173
9836991919,0cyclictest0-21swapper/207:15:192
9836991918,1cyclictest19067-21apt-get08:35:172
983699190,0cyclictest0-21swapper/207:45:182
983699190,0cyclictest0-21swapper/207:10:162
9832991919,0cyclictest0-21swapper/112:10:191
9832991919,0cyclictest0-21swapper/109:25:021
983299190,14cyclictest0-21swapper/110:57:141
9831991919,0cyclictest0-21swapper/012:10:190
9831991919,0cyclictest0-21swapper/011:05:170
9831991919,0cyclictest0-21swapper/010:35:160
9831991919,0cyclictest0-21swapper/009:40:170
9831991919,0cyclictest0-21swapper/008:30:010
9831991917,2cyclictest1401-21apt-get12:25:020
983199190,18cyclictest24929-21sendmail-msp11:00:000
9838991818,0cyclictest0-21swapper/311:10:163
9838991818,0cyclictest0-21swapper/308:10:163
9838991817,1cyclictest0-21swapper/308:40:173
983899180,17cyclictest0-21swapper/309:53:153
9836991818,0cyclictest0-21swapper/211:30:182
9836991818,0cyclictest0-21swapper/208:05:192
9836991818,0cyclictest0-21swapper/207:40:152
9836991817,1cyclictest0-21swapper/210:30:162
9836991817,1cyclictest0-21swapper/210:05:022
9836991816,1cyclictest8981-21idleruntime-cro12:40:002
983699180,0cyclictest0-21swapper/212:20:172
9832991818,0cyclictest32428-21apt-get11:15:171
9832991818,0cyclictest27376-21apt-get07:45:181
9832991818,0cyclictest0-21swapper/109:40:171
9832991818,0cyclictest0-21swapper/109:05:161
9832991818,0cyclictest0-21swapper/108:55:171
9832991818,0cyclictest0-21swapper/107:50:151
9832991818,0cyclictest0-21swapper/107:15:181
9831991818,0cyclictest0-21swapper/009:45:180
9831991818,0cyclictest0-21swapper/009:20:170
9831991818,0cyclictest0-21swapper/007:45:180
983199180,1cyclictest830-21turbostat09:05:010
983199180,1cyclictest830-21turbostat09:05:010
9838991717,0cyclictest0-21swapper/307:45:183
9836991717,0cyclictest8373-21apt-get10:25:162
9836991717,0cyclictest0-21swapper/212:00:172
9836991717,0cyclictest0-21swapper/207:50:152
9836991716,0cyclictest0-21swapper/208:00:172
983699170,16cyclictest2437-21apt-get11:20:162
983699170,16cyclictest19325-21cron11:55:002
9832991717,0cyclictest29284-21apt-get12:15:181
9832991717,0cyclictest15270-21apt-get07:20:161
9832991717,0cyclictest10028-21apt-get11:35:181
9832991717,0cyclictest0-21swapper/111:20:161
9832991717,0cyclictest0-21swapper/107:25:151
9832991715,1cyclictest607-21in:imuxsock10:25:001
983299170,0cyclictest0-21swapper/108:43:101
9831991717,0cyclictest1948-21ls08:00:000
9831991717,0cyclictest0-21swapper/008:40:170
9831991716,1cyclictest5979-21apt-get10:20:160
9831991716,0cyclictest11829-21apt-get08:20:150
9838991616,0cyclictest0-21swapper/312:35:153
9838991614,1cyclictest0-21swapper/310:45:273
983899160,15cyclictest2131-21cron11:20:013
9836991616,0cyclictest0-21swapper/212:05:182
9836991616,0cyclictest0-21swapper/210:15:182
983699160,8cyclictest0-21swapper/212:15:192
983699160,6cyclictest0-21swapper/207:55:162
9832991616,0cyclictest0-21swapper/112:30:171
9832991616,0cyclictest0-21swapper/108:15:161
9832991615,1cyclictest2225-21apt-get08:00:171
983299160,15cyclictest10489-21diskstats07:10:171
9831991616,0cyclictest0-21swapper/010:40:170
9831991616,0cyclictest0-21swapper/010:30:170
9831991616,0cyclictest0-21swapper/008:45:160
983199160,1cyclictest937-21munin-run10:10:010
9838991515,0cyclictest0-21swapper/307:25:153
9838991514,1cyclictest17233-21apt-get11:50:163
983899150,0cyclictest0-21swapper/311:05:173
9836991515,0cyclictest0-21swapper/211:40:172
9836991515,0cyclictest0-21swapper/210:35:162
9836991514,1cyclictest30032-21apt-get11:10:152
983699150,1cyclictest30160-21sed08:55:252
983699150,14cyclictest0-21swapper/209:05:552
9832991515,0cyclictest6873-21apt-get12:35:161
9832991515,0cyclictest31701-21apt-get12:20:171
9832991515,0cyclictest13173-21apt-get10:35:161
9832991515,0cyclictest0-21swapper/107:55:171
9832991514,1cyclictest0-21swapper/110:01:441
9832991514,1cyclictest0-21swapper/107:43:091
9831991515,0cyclictest0-21swapper/011:45:180
9831991514,1cyclictest18033-21irqstats11:50:200
9831991513,1cyclictest8978-21sh12:40:000
983199150,14cyclictest25924-21apt-get09:55:180
983199150,0cyclictest0-21swapper/010:46:540
9838991414,0cyclictest0-21swapper/308:56:183
9838991414,0cyclictest0-21swapper/307:55:153
983899140,14cyclictest0-21swapper/309:05:133
983899140,14cyclictest0-21swapper/308:45:313
983899140,14cyclictest0-21swapper/307:53:063
983899140,0cyclictest0-21swapper/308:15:013
983699140,14cyclictest0-21swapper/210:45:042
983699140,14cyclictest0-21swapper/209:58:162
983699140,14cyclictest0-21swapper/209:01:362
983699140,14cyclictest0-21swapper/209:01:362
983699140,14cyclictest0-21swapper/207:35:042
983699140,0cyclictest0-21swapper/207:30:302
9832991414,0cyclictest0-21swapper/108:20:081
9831991414,0cyclictest0-21swapper/012:00:300
9831991414,0cyclictest0-21swapper/008:55:160
9831991412,1cyclictest0-21swapper/008:00:170
983199140,14cyclictest0-21swapper/012:06:230
983199140,14cyclictest0-21swapper/011:40:300
983199140,14cyclictest0-21swapper/007:35:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional