You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-04 - 01:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sat Jan 03, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1335799260,0cyclictest0-21swapper/009:36:050
13357992422,1cyclictest0-21swapper/010:25:000
1335799240,23cyclictest8775-21idleruntime-cro10:20:000
1335799240,23cyclictest28028-21apt11:00:010
1335799240,23cyclictest0-21swapper/009:51:040
1336799232,20cyclictest607-21in:imuxsock08:45:003
13367992321,1cyclictest30240-21awk07:45:013
13367992320,2cyclictest29506-21idleruntime-cro08:50:003
1336799230,22cyclictest28004-21sh11:00:013
1336799230,22cyclictest26342-21cron09:50:013
1336799230,22cyclictest15686-21latency_hist11:40:013
1336299230,22cyclictest3200-21grep11:15:002
1336299230,1cyclictest3967-21munin-run10:10:012
13358992322,0cyclictest0-21swapper/108:51:011
1335899230,23cyclictest0-21swapper/107:55:021
1335899230,22cyclictest6355-21apt-get11:21:051
1335899230,22cyclictest30271-21perf07:45:011
1335899230,22cyclictest2640-21perf07:55:001
1335799230,2cyclictest6018-21sendmail-msp11:20:000
1335799230,22cyclictest0-21swapper/010:35:010
13367992222,0cyclictest25627-21apt-get12:01:053
13367992221,1cyclictest25681-21apt-get07:36:033
13367992220,1cyclictest18758-21cron09:35:003
1336799220,21cyclictest25350-21munin-run07:35:013
1336799220,21cyclictest25350-21munin-run07:35:013
1336799220,21cyclictest12663-21cron08:15:013
1336799220,21cyclictest0-21swapper/312:36:043
13362992220,1cyclictest23284-21apt-get07:31:032
13362992220,1cyclictest23284-21apt-get07:31:032
13362992220,1cyclictest21916-21apt-get09:41:042
13362992220,1cyclictest11909-21apt-get09:21:052
13362992220,1cyclictest10819-21munin-run11:30:012
1336299220,21cyclictest27676-21latency_hist12:05:002
1336299220,21cyclictest2649-21munin-run07:55:002
1336299220,21cyclictest0-21swapper/207:21:082
1336299220,1cyclictest26364-21ls09:50:012
13358992222,0cyclictest0-21swapper/107:21:081
13358992221,1cyclictest0-21swapper/111:06:021
13358992220,1cyclictest31512-21apt-get10:01:021
13358992220,1cyclictest16707-21apt-get09:31:031
1335899220,21cyclictest27448-21apt-get08:46:031
1335899220,21cyclictest24310-21apt-get09:46:051
1335899220,21cyclictest0-21swapper/111:26:041
1335899220,21cyclictest0-21swapper/109:35:021
13357992220,1cyclictest16377-21munin-run09:30:000
1335799220,22cyclictest0-21swapper/010:06:040
1335799220,21cyclictest6710-21perf09:10:010
1335799220,21cyclictest15736-21apt07:15:000
1335799220,1cyclictest0-21swapper/009:35:000
1335799220,1cyclictest0-21swapper/008:45:000
13367992120,1cyclictest0-21swapper/307:30:013
13367992119,1cyclictest3211-21apt-get12:21:053
13367992119,1cyclictest18417-21apt-get11:46:053
1336799210,21cyclictest0-21swapper/311:00:033
1336799210,20cyclictest0-21swapper/311:31:043
1336799210,20cyclictest0-21swapper/310:51:053
1336799210,20cyclictest0-21swapper/310:00:013
1336799210,1cyclictest0-21swapper/310:41:053
1336799210,0cyclictest22644-21apt-get08:36:083
1336799210,0cyclictest0-21swapper/311:11:033
13362992120,1cyclictest13603-21apt-get11:36:042
13362992120,1cyclictest0-21swapper/210:41:042
13362992120,0cyclictest0-21swapper/207:26:022
13362992119,1cyclictest10394-21apt-get12:36:042
13362992119,1cyclictest0-21swapper/212:00:002
1336299210,21cyclictest0-21swapper/209:11:032
1336299210,1cyclictest0-21swapper/209:36:062
1336299210,0cyclictest0-21swapper/208:41:042
13358992121,0cyclictest0-21swapper/108:41:041
13358992121,0cyclictest0-21swapper/108:16:021
13358992120,1cyclictest28410-21apt-get12:06:021
13358992120,1cyclictest16024-21perf10:35:011
13358992120,0cyclictest0-21swapper/111:36:041
13358992120,0cyclictest0-21swapper/111:11:031
13358992119,1cyclictest22876-21perf11:55:001
13358992119,1cyclictest0-21swapper/112:15:011
1335899210,20cyclictest20887-21apt-get07:26:021
1335899210,20cyclictest0-21swapper/108:26:041
1335899210,20cyclictest0-21swapper/108:05:001
1335899210,1cyclictest8183-21apt-get08:06:031
1335899210,0cyclictest0-21swapper/109:16:031
13357992121,0cyclictest0-21swapper/007:51:030
13357992120,0cyclictest0-21swapper/009:16:030
1335799210,21cyclictest0-21swapper/011:40:010
1335799210,21cyclictest0-21swapper/011:06:030
1335799210,21cyclictest0-21swapper/009:56:030
1335799210,1cyclictest21916-21apt-get09:41:040
1335799210,0cyclictest0-21swapper/011:01:020
1335799210,0cyclictest0-21swapper/007:26:020
13367992020,0cyclictest8183-21apt-get08:06:033
13367992020,0cyclictest0-21swapper/309:51:043
13367992020,0cyclictest0-21swapper/309:16:043
13367992020,0cyclictest0-21swapper/308:16:023
13367992018,1cyclictest8731-21cron10:20:003
13367992018,1cyclictest1894-21apt-get10:06:033
1336799200,20cyclictest0-21swapper/312:16:043
1336799200,20cyclictest0-21swapper/307:51:033
1336799200,0cyclictest4642-21apt-get09:06:023
1336799200,0cyclictest14306-21apt-get09:26:023
1336799200,0cyclictest0-21swapper/310:11:053
1336799200,0cyclictest0-21swapper/308:56:073
13362992020,0cyclictest0-21swapper/211:01:022
13362992020,0cyclictest0-21swapper/210:56:032
13362992020,0cyclictest0-21swapper/210:31:032
13362992020,0cyclictest0-21swapper/209:31:022
13362992020,0cyclictest0-21swapper/208:51:002
13362992020,0cyclictest0-21swapper/208:46:032
13362992020,0cyclictest0-21swapper/208:01:032
13362992020,0cyclictest0-21swapper/207:16:032
13362992019,1cyclictest0-21swapper/210:21:012
1336299200,20cyclictest0-21swapper/207:41:042
1336299200,0cyclictest0-21swapper/211:15:022
1336299200,0cyclictest0-21swapper/210:16:042
1336299200,0cyclictest0-21swapper/210:01:022
13358992020,0cyclictest0-21swapper/112:16:041
13358992020,0cyclictest0-21swapper/111:46:051
13358992020,0cyclictest0-21swapper/107:36:041
13358992020,0cyclictest0-21swapper/107:31:031
13358992020,0cyclictest0-21swapper/107:31:031
13358992018,1cyclictest11199-21apt-get11:31:031
13358992018,1cyclictest0-21swapper/110:21:011
13358992018,1cyclictest0-21swapper/110:15:001
13358992017,2cyclictest13981-21latency_hist09:25:011
1335899200,20cyclictest0-21swapper/110:51:041
1335899200,19cyclictest269752chrt12:01:131
1335899200,19cyclictest0-21swapper/110:41:051
1335899200,0cyclictest25954-21apt-get10:56:021
1335899200,0cyclictest23206-21apt-get11:56:041
1335899200,0cyclictest0-21swapper/111:01:021
1335899200,0cyclictest0-21swapper/110:26:031
13357992020,0cyclictest0-21swapper/012:36:040
13357992020,0cyclictest0-21swapper/012:05:020
13357992020,0cyclictest0-21swapper/011:51:040
13357992020,0cyclictest0-21swapper/010:46:040
13357992020,0cyclictest0-21swapper/010:01:020
13357992020,0cyclictest0-21swapper/009:21:050
13357992020,0cyclictest0-21swapper/008:46:030
13357992020,0cyclictest0-21swapper/008:26:030
13357992020,0cyclictest0-21swapper/007:31:030
13357992020,0cyclictest0-21swapper/007:31:030
13357992019,0cyclictest18769-21apt-get10:41:050
1335799200,20cyclictest0-21swapper/009:11:030
1335799200,0cyclictest764-21apt-get12:16:030
1335799200,0cyclictest0-21swapper/012:31:050
13367991919,0cyclictest0-21swapper/307:11:043
13362991919,0cyclictest13025-21apt-get08:16:032
13362991919,0cyclictest0-21swapper/212:16:042
13362991919,0cyclictest0-21swapper/209:26:012
13362991918,1cyclictest0-21swapper/212:30:012
13362991918,0cyclictest0-21swapper/212:06:022
13358991919,0cyclictest3205-21apt-get11:15:021
13358991919,0cyclictest0-21swapper/110:46:041
13358991919,0cyclictest0-21swapper/110:36:051
13358991917,1cyclictest0-21swapper/109:45:001
13357991919,0cyclictest0-21swapper/012:26:040
13357991919,0cyclictest0-21swapper/008:06:030
13357991919,0cyclictest0-21swapper/007:36:030
13357991917,1cyclictest18148-21perf07:20:000
13367991818,0cyclictest9100-21apt-get10:21:013
13367991818,0cyclictest5791-21apt-get08:01:043
13367991818,0cyclictest3258-21apt-get07:55:023
13367991818,0cyclictest0-21swapper/312:06:023
13367991818,0cyclictest0-21swapper/309:21:053
13367991818,0cyclictest0-21swapper/308:26:033
13367991816,1cyclictest606-21dbus-daemon11:55:003
13362991818,0cyclictest0-21swapper/209:01:032
13362991818,0cyclictest0-21swapper/208:56:072
13362991818,0cyclictest0-21swapper/208:36:082
13362991817,1cyclictest2634-21apt-get07:55:022
13362991816,1cyclictest807-21awk11:10:002
1336299180,1cyclictest17846-21apt-get08:26:042
1336299180,0cyclictest0-21swapper/211:46:042
13358991818,0cyclictest0-21swapper/112:21:051
13358991818,0cyclictest0-21swapper/109:51:041
13357991818,0cyclictest4291-21apt-get10:11:050
13357991818,0cyclictest0-21swapper/012:11:050
13357991818,0cyclictest0-21swapper/011:26:040
13357991817,1cyclictest3196-21turbostat11:15:000
13357991817,1cyclictest1880-21cron09:00:000
13357991817,0cyclictest0-21swapper/009:46:060
13362991717,0cyclictest15451-21apt-get08:21:022
13362991717,0cyclictest0-21swapper/211:51:042
13362991717,0cyclictest0-21swapper/209:06:032
13362991717,0cyclictest0-21swapper/208:06:032
13362991717,0cyclictest0-21swapper/207:46:032
13362991716,1cyclictest8003-21apt-get12:31:062
13358991717,0cyclictest16018-21apt-get11:41:021
13358991717,0cyclictest0-21swapper/109:56:031
13358991717,0cyclictest0-21swapper/109:06:031
13358991717,0cyclictest0-21swapper/108:11:041
13358991715,1cyclictest1917-21perf09:00:011
1335899170,16cyclictest5602-21apt-get12:26:041
1335899170,16cyclictest30593-21apt-get07:46:031
13357991717,0cyclictest13025-21apt-get08:16:030
13357991717,0cyclictest0-21swapper/011:41:020
13357991717,0cyclictest0-21swapper/010:26:030
13357991717,0cyclictest0-21swapper/008:51:010
13357991717,0cyclictest0-21swapper/008:31:030
1335799170,16cyclictest25291-21sendmail-msp12:00:000
1336799166,10cyclictest24120-21munin-node11:15:063
13367991616,0cyclictest6355-21apt-get11:21:053
13367991616,0cyclictest0-21swapper/310:26:033
13367991616,0cyclictest0-21swapper/307:21:083
13367991615,0cyclictest0-21swapper/310:46:043
13367991613,2cyclictest606-21dbus-daemon11:10:003
1336799161,14cyclictest163082sleep310:35:563
13362991616,0cyclictest0-21swapper/212:11:052
13362991616,0cyclictest0-21swapper/209:16:042
13358991616,0cyclictest0-21swapper/110:06:041
13358991616,0cyclictest0-21swapper/108:31:031
1335899160,15cyclictest17986-21chrt07:18:281
13357991616,0cyclictest15451-21apt-get08:21:020
13357991616,0cyclictest0-21swapper/011:46:040
13357991616,0cyclictest0-21swapper/011:21:050
13357991616,0cyclictest0-21swapper/009:01:030
1335799160,1cyclictest0-21swapper/011:35:010
13367991515,0cyclictest0-21swapper/311:41:023
13367991513,1cyclictest18162-21awk07:20:003
13367991513,1cyclictest0-21swapper/310:35:013
1336799150,14cyclictest23243-21mailstats09:41:123
13362991515,0cyclictest0-21swapper/207:36:032
13362991513,1cyclictest0-21swapper/212:21:132
1336299150,1cyclictest0-21swapper/210:11:142
1335899153,12cyclictest24120-21munin-node09:03:171
13358991515,0cyclictest0-21swapper/109:26:021
13358991515,0cyclictest0-21swapper/108:21:021
13358991514,1cyclictest0-21swapper/112:36:121
13357991515,0cyclictest0-21swapper/012:01:050
13357991515,0cyclictest0-21swapper/007:56:040
13357991515,0cyclictest0-21swapper/007:41:040
13357991513,1cyclictest0-21swapper/007:25:010
13367991414,0cyclictest0-21swapper/309:10:063
13367991414,0cyclictest0-21swapper/308:20:293
1336799140,14cyclictest0-21swapper/312:25:163
1336799140,14cyclictest0-21swapper/311:25:143
1336799140,14cyclictest0-21swapper/309:35:443
1336799140,14cyclictest0-21swapper/309:00:363
1336799140,0cyclictest0-21swapper/312:30:373
1336799140,0cyclictest0-21swapper/312:10:203
1336799140,0cyclictest0-21swapper/310:00:043
1336799140,0cyclictest0-21swapper/308:50:223
1336799140,0cyclictest0-21swapper/307:45:283
13362991414,0cyclictest0-21swapper/210:36:052
1336299140,14cyclictest0-21swapper/211:20:522
1336299140,14cyclictest0-21swapper/210:50:212
1336299140,14cyclictest0-21swapper/209:57:072
1336299140,14cyclictest0-21swapper/208:30:342
1336299140,14cyclictest0-21swapper/208:10:332
1336299140,14cyclictest0-21swapper/207:11:272
1336299140,0cyclictest0-21swapper/211:40:522
1336299140,0cyclictest0-21swapper/211:30:042
1336299140,0cyclictest0-21swapper/210:45:242
1336299140,0cyclictest0-21swapper/209:50:392
1335899140,14cyclictest0-21swapper/107:11:201
13357991414,0cyclictest0-21swapper/008:35:060
13357991414,0cyclictest0-21swapper/008:01:010
13357991414,0cyclictest0-21swapper/007:45:140
13357991413,1cyclictest23528-21/usr/sbin/munin10:51:050
1335799140,14cyclictest0-21swapper/012:20:150
1335799140,14cyclictest0-21swapper/010:36:060
1335799140,0cyclictest0-21swapper/008:11:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional