You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 06:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sat Jan 10, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1413299250,1cyclictest17793-21awk21:30:013
1412199250,24cyclictest10568-21idleruntime-cro21:15:010
1412199250,24cyclictest10568-21idleruntime-cro21:15:010
1412199240,23cyclictest11916-21idleruntime-cro23:30:000
14132992320,2cyclictest31502-21turbostat23:05:013
1413299230,2cyclictest14294-21perf22:30:003
1413299230,22cyclictest3709-21apt-get19:56:353
1413299230,0cyclictest0-21swapper/322:16:363
1412899230,2cyclictest28147-21latency_hist20:45:002
1412899230,22cyclictest9221-21apt-get00:30:022
1412899230,22cyclictest30552-21perf20:50:002
1412899230,22cyclictest29494-21apt-get23:01:382
1412899230,22cyclictest23700-21awk19:30:012
14124992321,1cyclictest26706-21cron00:00:011
14124992321,1cyclictest1943-21apt-get23:10:031
14124992320,2cyclictest26066-21cron19:35:011
1412499230,22cyclictest29792-21latency_hist21:55:011
1412199230,1cyclictest5747-21munin-run20:00:000
14132992220,1cyclictest9233-21dpkg00:30:013
14132992220,1cyclictest4707-21dump-pmu-power23:15:003
14132992220,1cyclictest0-21swapper/321:05:013
1413299220,2cyclictest606-21dbus-daemon20:50:003
1413299220,21cyclictest4046-21idleruntime-cro00:20:003
1413299220,21cyclictest14319-21cron23:35:013
1413299220,21cyclictest0-21swapper/323:36:353
1413299220,21cyclictest0-21swapper/321:56:353
1413299220,1cyclictest19137-21sh23:45:013
1413299220,0cyclictest0-21swapper/323:46:373
14128992220,2cyclictest5392-21perf21:05:012
1412899220,21cyclictest30920-21apt-get20:51:352
1412899220,21cyclictest22985-21apt-get21:41:352
1412899220,21cyclictest16854-21apt-get19:16:362
1412899220,21cyclictest14679-21apt-get22:31:362
1412899220,21cyclictest14311-21munin-run22:30:002
1412899220,21cyclictest0-21swapper/221:06:352
1412899220,21cyclictest0-21swapper/200:01:372
1412899220,1cyclictest0-21swapper/223:26:362
1412899220,0cyclictest0-21swapper/223:21:352
14124992221,0cyclictest32280-21apt-get23:06:371
14124992220,1cyclictest22595-21latency_hist21:40:001
14124992220,1cyclictest22318-21apt-get23:51:361
14124992220,1cyclictest0-21swapper/122:15:011
1412499220,21cyclictest7118-21dump-pmu-power23:20:001
1412499220,21cyclictest21308-21perf19:25:001
1412499220,21cyclictest15397-21latency_hist21:25:011
1412499220,21cyclictest0-21swapper/122:21:371
1412499220,1cyclictest0-21swapper/122:50:021
1412499220,0cyclictest0-21swapper/120:51:351
14121992220,1cyclictest28144-21perf20:45:000
1412199220,21cyclictest7208-21apt-get00:26:370
1412199220,21cyclictest28884-21apt-get19:41:360
1412199220,21cyclictest21670-21apt-get19:26:340
1412199220,21cyclictest14464-21apt-get19:11:370
1412199220,21cyclictest11907-21munin-plugin-st22:25:000
1412199220,1cyclictest18864-21cron19:20:000
1412199220,1cyclictest12007-21perf00:35:010
14132992121,0cyclictest19492-21apt-get22:41:373
14132992121,0cyclictest0-21swapper/322:36:373
14132992121,0cyclictest0-21swapper/320:06:343
14132992120,0cyclictest24067-21apt-get19:31:353
14132992120,0cyclictest0-21swapper/300:01:373
14132992119,1cyclictest29782-21awk21:55:013
14132992119,1cyclictest28141-21turbostat20:45:003
1413299210,20cyclictest2625-21apt-get22:06:363
1413299210,20cyclictest0-21swapper/319:46:373
1413299210,20cyclictest0-21swapper/319:41:363
1413299210,0cyclictest0-21swapper/323:06:363
1413299210,0cyclictest0-21swapper/320:21:353
1413299210,0cyclictest0-21swapper/320:11:353
14128992120,1cyclictest19274-21apt-get19:21:352
14128992120,1cyclictest1302-21apt-get19:51:362
14128992118,2cyclictest15360-21munin-run20:20:012
1412899210,21cyclictest0-21swapper/223:46:372
1412899210,21cyclictest0-21swapper/222:51:362
1412899210,20cyclictest6432-21apt-get00:25:022
1412899210,20cyclictest0-21swapper/220:56:362
1412899210,20cyclictest0-21swapper/220:01:362
1412899210,0cyclictest5104-21apt-get23:16:372
1412899210,0cyclictest0-21swapper/222:21:372
1412899210,0cyclictest0-21swapper/222:11:362
14124992121,0cyclictest0-21swapper/120:30:021
14124992120,1cyclictest0-21swapper/121:41:351
14124992119,1cyclictest2625-21apt-get22:06:351
14124992118,2cyclictest9435-21idleruntime-cro22:20:011
1412499210,21cyclictest0-21swapper/120:56:361
1412499210,20cyclictest30170-21apt-get21:56:351
1412499210,20cyclictest17131-21apt-get23:41:361
1412499210,0cyclictest0-21swapper/100:25:021
1412199210,21cyclictest0-21swapper/021:05:020
1412199210,20cyclictest27776-21apt-get21:51:360
1412199210,20cyclictest0-21swapper/019:21:350
1412199210,0cyclictest0-21swapper/020:31:370
14132992020,0cyclictest0-21swapper/300:36:373
14132992019,0cyclictest0-21swapper/320:56:363
14132992018,1cyclictest13000-21perf21:20:013
14132992017,2cyclictest16474-21turbostat19:15:003
1413299200,20cyclictest0-21swapper/319:21:353
1413299200,19cyclictest26700-21cron00:00:013
1413299200,19cyclictest23901-21apt-get22:50:023
1413299200,0cyclictest21888-21apt-get22:46:373
1413299200,0cyclictest0-21swapper/320:01:363
1413299200,0cyclictest0-21swapper/300:21:363
1413299200,0cyclictest0-21swapper/300:06:373
14128992020,0cyclictest8518-21apt-get20:06:352
14128992020,0cyclictest26467-21apt-get19:36:362
14128992020,0cyclictest0-21swapper/223:41:352
14128992020,0cyclictest0-21swapper/223:06:362
14128992020,0cyclictest0-21swapper/220:36:352
14128992020,0cyclictest0-21swapper/219:11:372
14128992019,1cyclictest0-21swapper/222:16:362
14128992019,1cyclictest0-21swapper/220:30:022
14128992019,0cyclictest0-21swapper/219:56:352
1412899200,0cyclictest12390-21apt-get00:36:362
14124992020,0cyclictest9991-21apt-get00:31:371
14124992020,0cyclictest29620-21apt-get00:06:371
14124992020,0cyclictest0-21swapper/123:01:381
14124992020,0cyclictest0-21swapper/122:01:361
14124992020,0cyclictest0-21swapper/121:46:351
14124992020,0cyclictest0-21swapper/120:40:021
14124992020,0cyclictest0-21swapper/120:06:351
14124992020,0cyclictest0-21swapper/100:01:371
14124992018,1cyclictest12917-21idleruntime-cro20:15:001
14124992018,1cyclictest0-21swapper/122:35:001
1412499200,20cyclictest0-21swapper/100:36:371
14121992020,0cyclictest0-21swapper/023:56:350
14121992020,0cyclictest0-21swapper/023:05:020
14121992020,0cyclictest0-21swapper/022:01:350
14121992020,0cyclictest0-21swapper/020:36:360
14121992020,0cyclictest0-21swapper/000:21:360
14121992020,0cyclictest0-21swapper/000:11:350
14121992019,1cyclictest14304-21turbostat22:30:000
1412199200,20cyclictest0-21swapper/023:41:360
1412199200,20cyclictest0-21swapper/021:26:350
1412199200,19cyclictest8132-21latency_hist20:05:000
1412199200,0cyclictest0-21swapper/022:46:370
1412199200,0cyclictest0-21swapper/021:46:340
14132991919,0cyclictest0-21swapper/323:16:373
14132991919,0cyclictest0-21swapper/322:11:353
14132991919,0cyclictest0-21swapper/322:01:353
14132991919,0cyclictest0-21swapper/321:06:353
14132991919,0cyclictest0-21swapper/320:26:343
14132991919,0cyclictest0-21swapper/319:16:363
14132991919,0cyclictest0-21swapper/300:30:023
14132991918,1cyclictest13308-21apt-get20:16:373
14132991916,2cyclictest16713-21awk22:35:003
14128991919,0cyclictest0-21swapper/223:36:362
14128991919,0cyclictest0-21swapper/223:11:372
14128991919,0cyclictest0-21swapper/222:46:372
14128991919,0cyclictest0-21swapper/221:56:352
14128991919,0cyclictest0-21swapper/221:51:362
14128991919,0cyclictest0-21swapper/221:36:362
14128991919,0cyclictest0-21swapper/220:21:352
14128991918,1cyclictest0-21swapper/200:16:362
1412899190,0cyclictest0-21swapper/220:11:362
14124991919,0cyclictest31279-21apt-get19:46:371
14124991919,0cyclictest0-21swapper/120:26:341
14124991919,0cyclictest0-21swapper/119:16:361
14124991919,0cyclictest0-21swapper/100:16:371
14124991917,1cyclictest5729-21perf20:00:001
14124991917,1cyclictest17803-21munin-run21:30:011
14124991916,2cyclictest11936-21munin-plugin-st23:30:001
14121991919,0cyclictest8518-21apt-get20:06:350
14121991919,0cyclictest24676-21apt-get22:51:360
14121991919,0cyclictest0-21swapper/023:21:350
14121991919,0cyclictest0-21swapper/022:16:360
14121991918,0cyclictest0-21swapper/023:46:370
1412199190,19cyclictest0-21swapper/023:51:360
14132991818,0cyclictest0-21swapper/320:31:373
14132991818,0cyclictest0-21swapper/300:11:353
14132991817,1cyclictest9834-21apt-get22:21:373
14128991818,0cyclictest0-21swapper/223:51:372
14128991818,0cyclictest0-21swapper/221:46:342
14128991818,0cyclictest0-21swapper/219:31:352
14128991817,1cyclictest4418-21apt-get00:21:352
14128991817,1cyclictest32587-21apt-get22:01:362
14124991818,0cyclictest0-21swapper/121:11:351
14124991818,0cyclictest0-21swapper/121:11:351
14124991818,0cyclictest0-21swapper/119:36:361
14124991817,1cyclictest7519-21apt-get23:21:351
14124991816,1cyclictest313-21systemd-journal22:40:001
14124991816,1cyclictest17750-21awk20:25:011
14121991818,0cyclictest5019-21apt-get22:11:360
14121991818,0cyclictest0-21swapper/023:01:380
14121991818,0cyclictest0-21swapper/021:36:360
14121991818,0cyclictest0-21swapper/020:26:340
14121991818,0cyclictest0-21swapper/000:36:370
14121991818,0cyclictest0-21swapper/000:06:360
14121991817,1cyclictest774-21runrttasks22:31:360
14121991817,1cyclictest10914-21apt-get20:11:360
14121991817,1cyclictest0-21swapper/020:50:000
14121991816,1cyclictest5104-21apt-get23:16:370
14132991717,0cyclictest0-21swapper/319:36:353
14132991716,1cyclictest0-21swapper/320:51:353
14132991716,0cyclictest25379-21apt-get21:46:343
14128991717,0cyclictest0-21swapper/222:41:362
14128991717,0cyclictest0-21swapper/221:26:352
14128991717,0cyclictest0-21swapper/219:46:372
14128991715,1cyclictest26733-21awk00:00:012
14124991717,0cyclictest13308-21apt-get20:16:371
14124991717,0cyclictest0-21swapper/123:36:361
14124991717,0cyclictest0-21swapper/122:46:371
14124991717,0cyclictest0-21swapper/122:41:371
14124991717,0cyclictest0-21swapper/121:06:351
14124991717,0cyclictest0-21swapper/120:46:361
14124991717,0cyclictest0-21swapper/119:41:361
14124991715,1cyclictest3351-21perf19:55:011
14121991717,0cyclictest27223-21apt-get00:01:370
14121991717,0cyclictest0-21swapper/020:51:350
14121991716,1cyclictest892-21apt-get20:56:360
14121991716,0cyclictest0-21swapper/021:41:350
1412199170,15cyclictest0-21swapper/021:56:440
14128991616,0cyclictest0-21swapper/200:06:372
14128991615,0cyclictest0-21swapper/222:06:352
1412499163,12cyclictest793-21lldpd21:01:031
14124991614,1cyclictest23316-21apt-get20:36:361
14124991614,1cyclictest22310-21hddtemp_smartct19:26:371
1412499160,15cyclictest21534-21cron23:50:011
1412499160,0cyclictest0-21swapper/100:24:551
14121991616,0cyclictest12311-21apt-get23:31:360
14121991614,1cyclictest5404-21munin-run21:05:010
14121991614,1cyclictest0-21swapper/020:16:000
1412199160,15cyclictest31826-21perl19:46:390
14132991514,1cyclictest13374-21apt-get21:21:363
1413299150,3cyclictest0-21swapper/321:41:353
14128991515,0cyclictest17076-21apt-get22:36:372
14128991515,0cyclictest10958-21apt-get21:16:372
14128991515,0cyclictest0-21swapper/223:31:362
14124991512,2cyclictest29085-21latency_hist23:00:011
1412199154,11cyclictest16650-21kworker/0:123:11:230
14121991515,0cyclictest0-21swapper/019:51:360
14121991515,0cyclictest0-21swapper/019:36:360
14121991514,1cyclictest17718-21hddtemp_smartct22:36:390
14132991414,0cyclictest0-21swapper/321:10:533
14132991414,0cyclictest0-21swapper/321:10:533
1413299140,14cyclictest0-21swapper/323:25:073
1413299140,14cyclictest0-21swapper/322:55:193
1413299140,14cyclictest0-21swapper/321:36:223
1413299140,14cyclictest0-21swapper/321:30:563
1413299140,14cyclictest0-21swapper/319:50:083
1413299140,0cyclictest0-21swapper/323:50:083
1413299140,0cyclictest0-21swapper/323:20:103
1413299140,0cyclictest0-21swapper/320:35:323
1413299140,0cyclictest0-21swapper/319:25:143
1412899140,14cyclictest0-21swapper/221:30:462
1412899140,14cyclictest0-21swapper/221:20:042
1412899140,14cyclictest0-21swapper/219:42:122
1412899140,0cyclictest0-21swapper/222:55:092
1412899140,0cyclictest0-21swapper/221:10:112
1412899140,0cyclictest0-21swapper/221:10:112
1412899140,0cyclictest0-21swapper/200:10:212
1412499140,14cyclictest0-21swapper/123:30:021
1412499140,14cyclictest0-21swapper/122:25:201
1412499140,0cyclictest0-21swapper/121:30:301
1412499140,0cyclictest0-21swapper/121:15:061
1412499140,0cyclictest0-21swapper/120:00:371
14121991414,0cyclictest0-21swapper/022:40:240
1412199140,14cyclictest0-21swapper/023:35:350
1412199140,14cyclictest0-21swapper/022:55:480
1412199140,14cyclictest0-21swapper/022:05:200
1412199140,14cyclictest0-21swapper/021:30:320
1412199140,14cyclictest0-21swapper/019:31:050
1412199140,14cyclictest0-21swapper/000:15:150
1412199140,0cyclictest0-21swapper/021:21:140
1412199140,0cyclictest0-21swapper/021:15:110
1412199140,0cyclictest0-21swapper/020:20:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional