You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 05:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Dec 09, 2025 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3114399250,0cyclictest0-21swapper/220:58:392
31142992523,1cyclictest22-21ksoftirqd/121:50:011
3114299249,14cyclictest22-21ksoftirqd/119:18:391
31142992422,1cyclictest4404-21apt-get22:40:021
31142992420,3cyclictest7360-21munin-run21:40:001
3114199242,21cyclictest61550irq/126-eno120:58:390
31144992322,1cyclictest4462-21apt-get19:23:343
31144992321,1cyclictest27988-21cron20:10:003
3114399230,23cyclictest0-21swapper/200:20:012
3114399230,22cyclictest30045-21cron21:20:002
31142992321,1cyclictest10868-21perf20:40:001
3114299230,0cyclictest6448-21apt-get23:53:381
3114299230,0cyclictest0-21swapper/121:53:391
31141992323,0cyclictest0-21swapper/020:28:370
31141992321,1cyclictest18075-21munin-plugin-st20:55:000
3114199230,22cyclictest313-21systemd-journal20:20:010
3114199230,22cyclictest21042-21apt-get00:20:000
3114199230,22cyclictest15965-21idleruntime-cro19:44:590
3114199230,22cyclictest0-21swapper/000:28:380
3114199230,1cyclictest0-21swapper/022:05:000
31144992220,1cyclictest7356-21perf21:40:003
31144992220,1cyclictest20393-21dump-pmu-power00:20:003
31144992220,1cyclictest0-21swapper/322:45:003
31144992220,1cyclictest0-21swapper/319:18:383
31144992219,2cyclictest23851-21perf23:20:003
3114499220,2cyclictest14514-21cron21:55:003
3114499220,21cyclictest29509-21apt-get22:28:373
3114499220,21cyclictest13832-21apt-get23:00:013
3114499220,21cyclictest1-21systemd23:59:593
3114499220,21cyclictest0-21swapper/322:18:383
3114499220,21cyclictest0-21swapper/321:48:383
3114499220,21cyclictest0-21swapper/320:58:393
3114499220,21cyclictest0-21swapper/320:23:363
3114499220,21cyclictest0-21swapper/300:38:383
3114499220,1cyclictest6448-21apt-get23:53:383
3114499220,1cyclictest31047-21perf23:35:003
3114499220,1cyclictest24390-21apt-get23:23:403
3114499220,1cyclictest23201-21perf20:00:003
3114499220,1cyclictest19046-21munin-run23:10:003
3114499220,1cyclictest0-21swapper/320:30:003
3114399220,2cyclictest8749-21cron19:30:002
3114399220,21cyclictest766-21snmpd22:13:372
3114399220,21cyclictest593-21dbus-daemon23:44:592
31142992220,1cyclictest13857-21awk23:00:001
3114299220,2cyclictest20485-21latency_hist21:00:001
3114299220,21cyclictest21360-21runrttasks00:23:401
3114299220,21cyclictest19370-21grep22:05:001
3114299220,21cyclictest11430-21awk22:55:001
3114299220,21cyclictest0-21swapper/122:28:371
3114299220,21cyclictest0-21swapper/121:08:371
3114299220,21cyclictest0-21swapper/120:48:371
3114299220,1cyclictest0-21swapper/123:03:391
3114199220,21cyclictest6611-21sh22:45:000
3114199220,21cyclictest13854-21munin-run23:00:000
3114199220,21cyclictest0-21swapper/023:13:380
3114199220,21cyclictest0-21swapper/021:58:370
3114199220,21cyclictest0-21swapper/019:38:360
3114199220,1cyclictest6917-21apt-get19:28:380
31144992121,0cyclictest0-21swapper/300:10:013
31144992120,1cyclictest17502-21apt-get22:03:383
31144992120,1cyclictest0-21swapper/322:48:383
31144992119,1cyclictest18396-21munin-run19:50:003
3114499210,20cyclictest13799-21apt-get20:48:373
3114499210,20cyclictest0-21swapper/322:08:393
3114499210,20cyclictest0-21swapper/320:53:373
3114499210,1cyclictest23412-21apt-get21:08:373
3114499210,1cyclictest13566-21idleruntime-cro19:40:003
3114499210,0cyclictest0-21swapper/321:43:403
3114399210,21cyclictest25280-21perf21:10:012
3114399210,20cyclictest8988-21apt-get20:38:392
3114399210,20cyclictest16196-21apt-get20:53:362
3114399210,20cyclictest0-21swapper/222:28:372
3114399210,1cyclictest16526-21apt-get19:48:392
3114399210,0cyclictest0-21swapper/200:13:382
31142992121,0cyclictest0-21swapper/120:13:371
31142992121,0cyclictest0-21swapper/119:38:361
31142992120,1cyclictest22-21ksoftirqd/120:03:381
31142992120,1cyclictest0-21swapper/100:33:381
31142992119,2cyclictest22-21ksoftirqd/119:28:381
31142992119,1cyclictest26256-21grep23:25:011
31142992119,1cyclictest19042-21perf23:10:001
3114299210,20cyclictest0-21swapper/121:33:391
3114299210,20cyclictest0-21swapper/120:53:361
3114299210,20cyclictest0-21swapper/120:18:371
3114299210,0cyclictest11350-21apt-get00:03:381
3114299210,0cyclictest0-21swapper/100:18:371
31141992121,0cyclictest28212-21apt-get21:18:380
31141992121,0cyclictest0-21swapper/020:08:360
31141992119,1cyclictest3927-21apt19:20:000
3114199210,21cyclictest1057-21sendmail-msp23:40:000
3114199210,21cyclictest0-21swapper/021:43:390
3114199210,20cyclictest0-21swapper/023:48:390
3114199210,20cyclictest0-21swapper/022:38:370
3114199210,0cyclictest26786-21apt-get23:28:400
3114199210,0cyclictest0-21swapper/023:23:390
3114199210,0cyclictest0-21swapper/020:43:370
31144992020,0cyclictest26148-21apt-get00:33:383
31144992020,0cyclictest0-21swapper/323:38:383
31144992020,0cyclictest0-21swapper/322:38:373
31144992020,0cyclictest0-21swapper/320:38:393
31144992020,0cyclictest0-21swapper/300:23:393
31144992019,1cyclictest0-21swapper/321:28:413
31144992019,0cyclictest31663-21apt-get19:13:383
31144992018,1cyclictest0-21swapper/319:33:373
3114499200,0cyclictest9566-21apt-get22:53:383
3114499200,0cyclictest0-21swapper/322:58:383
3114499200,0cyclictest0-21swapper/320:05:003
31143992020,0cyclictest11962-21apt-get22:58:382
31143992020,0cyclictest0-21swapper/223:48:392
31143992020,0cyclictest0-21swapper/223:28:402
31143992020,0cyclictest0-21swapper/223:13:382
31143992020,0cyclictest0-21swapper/222:23:372
31143992020,0cyclictest0-21swapper/222:23:372
31143992020,0cyclictest0-21swapper/221:58:372
31143992020,0cyclictest0-21swapper/220:28:372
31143992020,0cyclictest0-21swapper/220:18:362
31143992020,0cyclictest0-21swapper/220:13:382
31143992020,0cyclictest0-21swapper/219:58:362
31143992020,0cyclictest0-21swapper/219:23:352
31143992020,0cyclictest0-21swapper/219:15:022
31143992019,1cyclictest1903-21apt-get22:38:372
31143992018,1cyclictest15974-21awk19:45:002
3114399200,20cyclictest0-21swapper/223:53:382
3114399200,20cyclictest0-21swapper/223:03:392
3114399200,20cyclictest0-21swapper/222:40:022
3114399200,20cyclictest0-21swapper/200:38:382
3114399200,20cyclictest0-21swapper/200:18:382
3114399200,0cyclictest0-21swapper/223:38:382
3114399200,0cyclictest0-21swapper/221:43:402
31142992020,0cyclictest22295-21apt-get22:13:371
31142992020,0cyclictest0-21swapper/123:58:391
31142992020,0cyclictest0-21swapper/123:43:381
31142992020,0cyclictest0-21swapper/123:28:391
31142992020,0cyclictest0-21swapper/122:33:371
31142992020,0cyclictest0-21swapper/122:23:371
31142992020,0cyclictest0-21swapper/122:23:371
31142992020,0cyclictest0-21swapper/122:08:391
31142992020,0cyclictest0-21swapper/121:18:391
31142992020,0cyclictest0-21swapper/121:13:371
31142992020,0cyclictest0-21swapper/120:28:371
31142992020,0cyclictest0-21swapper/120:23:361
31142992020,0cyclictest0-21swapper/119:23:361
3114299200,0cyclictest0-21swapper/123:48:391
3114299200,0cyclictest0-21swapper/100:28:381
31141992020,0cyclictest14776-21apt-get23:03:390
31141992020,0cyclictest0-21swapper/023:58:390
31141992020,0cyclictest0-21swapper/023:33:370
31141992020,0cyclictest0-21swapper/023:08:390
31141992020,0cyclictest0-21swapper/022:23:370
31141992020,0cyclictest0-21swapper/022:23:370
31141992020,0cyclictest0-21swapper/021:48:380
31141992020,0cyclictest0-21swapper/019:53:360
31141992020,0cyclictest0-21swapper/000:35:000
31141992018,1cyclictest23208-21cut20:00:000
31141992018,1cyclictest14544-21perf21:55:000
3114199200,0cyclictest0-21swapper/022:48:380
3114199200,0cyclictest0-21swapper/020:23:360
3114199200,0cyclictest0-21swapper/000:18:370
31144991919,0cyclictest22295-21apt-get22:13:373
31144991918,0cyclictest25810-21apt-get21:13:373
31144991918,0cyclictest0-21swapper/323:13:373
31143991919,0cyclictest0-21swapper/223:58:392
31143991919,0cyclictest0-21swapper/221:53:392
31143991919,0cyclictest0-21swapper/220:33:382
31143991919,0cyclictest0-21swapper/219:53:372
31143991919,0cyclictest0-21swapper/200:28:382
31143991918,1cyclictest27681-21munin-run21:15:002
31143991918,1cyclictest11709-21apt-get19:38:362
3114399190,0cyclictest30627-21apt-get21:23:372
31142991919,0cyclictest0-21swapper/123:38:391
31142991919,0cyclictest0-21swapper/120:08:361
31142991919,0cyclictest0-21swapper/119:43:401
31142991919,0cyclictest0-21swapper/100:38:381
31142991919,0cyclictest0-21swapper/100:08:381
31142991917,2cyclictest0-21swapper/121:58:371
31141991919,0cyclictest0-21swapper/022:08:390
31141991919,0cyclictest0-21swapper/021:33:390
31141991919,0cyclictest0-21swapper/021:23:370
31141991919,0cyclictest0-21swapper/021:03:350
31141991919,0cyclictest0-21swapper/000:03:380
31141991918,0cyclictest0-21swapper/020:10:020
31144991818,0cyclictest0-21swapper/321:18:383
31144991817,1cyclictest23752-21apt-get00:28:383
31144991817,1cyclictest14131-21apt-get19:43:403
31144991816,1cyclictest20793-21wc19:55:003
31143991818,0cyclictest21013-21apt-get21:03:372
31143991818,0cyclictest11403-21apt-get20:43:372
31143991818,0cyclictest0-21swapper/223:23:402
31143991818,0cyclictest0-21swapper/220:48:372
31143991818,0cyclictest0-21swapper/200:33:382
31143991817,1cyclictest19365-21apt22:04:592
3114399180,17cyclictest1329-21apt-get20:23:362
31142991818,0cyclictest0-21swapper/119:13:381
31142991817,1cyclictest22-21ksoftirqd/119:35:011
31141991818,0cyclictest0-21swapper/023:53:380
31141991818,0cyclictest0-21swapper/022:13:370
31141991818,0cyclictest0-21swapper/021:13:370
31141991818,0cyclictest0-21swapper/021:08:380
31141991818,0cyclictest0-21swapper/000:08:370
31141991817,1cyclictest26592-21awk22:20:000
31141991816,1cyclictest0-21swapper/019:23:350
31141991816,1cyclictest0-21swapper/000:40:000
31144991717,0cyclictest31333-21apt-get20:18:353
31144991717,0cyclictest0-21swapper/322:23:373
31144991717,0cyclictest0-21swapper/322:23:373
31144991717,0cyclictest0-21swapper/320:43:383
3114499170,16cyclictest0-21swapper/320:13:453
31143991717,0cyclictest0-21swapper/222:18:382
31143991717,0cyclictest0-21swapper/222:08:392
31142991717,0cyclictest0-21swapper/122:38:361
31142991717,0cyclictest0-21swapper/122:18:381
31142991717,0cyclictest0-21swapper/121:43:401
31141991717,0cyclictest0-21swapper/021:28:410
31141991717,0cyclictest0-21swapper/019:48:380
31141991715,1cyclictest25614-21munin-run20:05:000
31144991616,0cyclictest0-21swapper/322:33:373
31144991615,0cyclictest1589-21apt-get23:43:383
31143991616,0cyclictest31916-21apt-get22:33:372
31143991616,0cyclictest0-21swapper/200:03:382
31143991615,1cyclictest19041-21awk23:10:002
31143991614,1cyclictest26741-21grep20:08:392
3114299167,8cyclictest22-21ksoftirqd/100:13:401
3114299164,11cyclictest22-21ksoftirqd/119:55:001
31142991616,0cyclictest0-21swapper/123:18:381
31142991615,0cyclictest0-21swapper/121:23:371
31141991616,0cyclictest0-21swapper/022:53:380
31141991616,0cyclictest0-21swapper/022:28:360
31141991616,0cyclictest0-21swapper/021:38:370
31141991615,0cyclictest0-21swapper/020:48:370
3114199161,14cyclictest0-21swapper/000:11:530
31144991515,0cyclictest0-21swapper/323:28:403
31144991515,0cyclictest0-21swapper/319:28:373
31144991515,0cyclictest0-21swapper/300:03:383
31143991515,0cyclictest7169-21apt-get22:48:382
31143991513,1cyclictest31631-21/usr/sbin/munin19:13:402
3114399150,14cyclictest0-21swapper/221:38:392
31142991515,0cyclictest0-21swapper/119:48:391
31142991514,1cyclictest211rcuc/123:33:441
31142991514,1cyclictest0-21swapper/123:13:441
31142991513,1cyclictest7842-21grep20:33:471
31142991512,2cyclictest22809-21cat19:58:441
3114299150,1cyclictest21816-21irqstats21:03:401
31141991515,0cyclictest0-21swapper/023:18:370
31141991514,1cyclictest0-21swapper/020:38:390
3114199150,0cyclictest0-21swapper/023:43:380
31144991414,0cyclictest0-21swapper/321:02:213
31144991413,0cyclictest15081-21apt-get21:58:373
3114499140,14cyclictest0-21swapper/323:49:123
3114499140,14cyclictest0-21swapper/321:21:253
3114499140,14cyclictest0-21swapper/320:30:513
3114499140,0cyclictest0-21swapper/321:31:153
3114399140,14cyclictest0-21swapper/223:32:202
3114399140,14cyclictest0-21swapper/223:15:582
3114399140,14cyclictest0-21swapper/221:32:202
3114399140,14cyclictest0-21swapper/220:02:482
3114399140,14cyclictest0-21swapper/200:05:042
3114399140,0cyclictest0-21swapper/222:50:442
3114399140,0cyclictest0-21swapper/221:46:132
3114399140,0cyclictest0-21swapper/221:27:112
3114299140,14cyclictest0-21swapper/122:46:201
3114299140,0cyclictest0-21swapper/120:40:011
31141991414,0cyclictest0-21swapper/019:13:220
3114199140,14cyclictest0-21swapper/022:33:340
3114199140,14cyclictest0-21swapper/020:30:310
3114199140,14cyclictest0-21swapper/019:31:100
31144991313,0cyclictest0-21swapper/300:11:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional