You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 03:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Mon Dec 08, 2025 00:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2480999260,0cyclictest0-21swapper/320:58:333
2480999250,20cyclictest0-21swapper/323:43:323
24809992323,0cyclictest16082-21apt-get00:23:343
24809992321,1cyclictest28655-21grep23:40:003
2480999230,22cyclictest0-21swapper/322:45:013
2480899230,22cyclictest9902-21taskset23:03:342
24802992320,2cyclictest7181-21lockfile-remove19:40:001
24797992320,2cyclictest13239-21perf21:00:000
2479799230,2cyclictest8979-21cat23:00:000
2479799230,2cyclictest8979-21cat23:00:000
2479799230,22cyclictest29366-21apt-get20:28:330
2479799230,22cyclictest0-21swapper/019:53:310
2479799230,22cyclictest0-21swapper/000:38:330
2479799230,22cyclictest0-21swapper/000:38:330
24809992222,0cyclictest20873-21apt-get00:33:363
2480999220,21cyclictest26237-21latency_hist23:35:003
2480999220,21cyclictest14359-21grep19:55:003
2480999220,21cyclictest0-21swapper/322:13:313
2480999220,21cyclictest0-21swapper/321:18:333
2480999220,21cyclictest0-21swapper/319:23:323
2480999220,21cyclictest0-21swapper/300:38:333
2480999220,21cyclictest0-21swapper/300:38:333
2480899221,0cyclictest6167-21apt-get22:55:022
2480899221,0cyclictest6167-21apt-get22:55:022
2480899220,21cyclictest6572-21apt-get20:48:332
2480899220,21cyclictest0-21swapper/200:13:342
2480899220,21cyclictest0-21swapper/200:05:012
2480899220,1cyclictest28824-21dump-pmu-power20:25:002
2480899220,1cyclictest21587-21cron20:09:592
24802992222,0cyclictest0-21swapper/120:33:321
24802992220,1cyclictest13123-21turbostat00:15:001
2480299220,22cyclictest13723-21munin-run22:05:001
2480299220,21cyclictest23267-21apt-get00:38:331
2480299220,21cyclictest23267-21apt-get00:38:331
2480299220,21cyclictest0-21swapper/121:58:321
2480299220,21cyclictest0-21swapper/121:38:351
2480299220,1cyclictest11972-21cut19:50:001
2480299220,1cyclictest0-21swapper/120:30:001
24797992222,0cyclictest0-21swapper/021:53:330
24797992220,1cyclictest141rcuc/022:30:010
24797992220,1cyclictest141rcuc/019:40:000
2479799220,21cyclictest10714-21ls00:10:000
2479799220,21cyclictest0-21swapper/020:05:010
2479799220,21cyclictest0-21swapper/019:28:320
2479799220,1cyclictest11844-21apt-get22:03:340
24809992121,0cyclictest0-21swapper/319:38:323
24809992121,0cyclictest0-21swapper/300:13:343
24809992120,1cyclictest8983-21apt-get23:00:023
24809992120,1cyclictest8868-21apt-get00:08:333
24809992119,2cyclictest8435-21turbostat.cron20:50:003
24809992119,1cyclictest27120-21cron19:15:013
24809992118,2cyclictest313-21systemd-journal23:20:003
2480999210,21cyclictest0-21swapper/323:08:323
2480999210,20cyclictest0-21swapper/322:38:343
2480999210,20cyclictest0-21swapper/322:18:333
2480999210,20cyclictest0-21swapper/320:03:343
2480999210,20cyclictest0-21swapper/319:48:333
2480999210,20cyclictest0-21swapper/319:45:003
2480999210,1cyclictest0-21swapper/322:58:333
2480999210,1cyclictest0-21swapper/322:58:333
2480999210,0cyclictest1561-21apt-get23:53:323
2480999210,0cyclictest1040-21apt-get21:43:333
24808992121,0cyclictest0-21swapper/222:48:332
24808992121,0cyclictest0-21swapper/222:33:332
24808992121,0cyclictest0-21swapper/222:23:332
24808992120,1cyclictest21028-21apt-get21:18:332
24808992119,1cyclictest18550-21dump-pmu-power22:15:002
24808992119,1cyclictest11365-21apt-get20:58:322
2480899210,20cyclictest3968-21apt-get23:58:342
2480899210,20cyclictest23902-21apt-get22:28:322
2480899210,20cyclictest0-21swapper/223:23:322
2480899210,0cyclictest0-21swapper/222:40:012
2480899210,0cyclictest0-21swapper/221:53:332
2480899210,0cyclictest0-21swapper/200:18:322
24802992121,0cyclictest13663-21apt-get00:18:321
24802992119,1cyclictest3619-21latency_hist20:40:001
24802992119,1cyclictest3619-21latency_hist20:40:001
24802992119,1cyclictest15550-21basename19:58:351
2480299210,20cyclictest23446-21apt-get21:23:331
2480299210,20cyclictest0-21swapper/123:38:371
2480299210,0cyclictest13780-21apt-get21:03:321
2480299210,0cyclictest0-21swapper/123:58:341
2480299210,0cyclictest0-21swapper/100:23:331
24797992121,0cyclictest8112-21apt-get21:55:010
24797992121,0cyclictest30569-21apt-get22:40:010
24797992121,0cyclictest18476-21apt-get00:28:330
24797992121,0cyclictest0-21swapper/021:45:020
24797992120,0cyclictest0-21swapper/020:43:350
24797992120,0cyclictest0-21swapper/019:43:340
2479799210,21cyclictest0-21swapper/021:28:330
2479799210,20cyclictest29543-21apt-get19:20:020
2479799210,20cyclictest0-21swapper/019:13:320
2479799210,1cyclictest0-21swapper/023:28:330
2479799210,0cyclictest0-21swapper/023:08:320
2479799210,0cyclictest0-21swapper/021:13:360
2479799210,0cyclictest0-21swapper/020:53:330
2479799210,0cyclictest0-21swapper/019:18:340
24809992020,0cyclictest0-21swapper/322:03:343
24809992020,0cyclictest0-21swapper/300:18:323
24809992019,0cyclictest14691-21apt-get23:13:333
24809992018,2cyclictest0-21swapper/319:58:323
24809992018,1cyclictest31761-21apt-get20:33:323
24809992018,1cyclictest0-21swapper/323:58:343
2480999200,20cyclictest0-21swapper/322:43:323
2480999200,0cyclictest0-21swapper/323:28:333
2480999200,0cyclictest0-21swapper/321:55:013
2480999200,0cyclictest0-21swapper/321:23:333
2480999200,0cyclictest0-21swapper/319:28:323
2480999200,0cyclictest0-21swapper/319:18:343
24808992020,0cyclictest0-21swapper/222:18:332
24808992020,0cyclictest0-21swapper/221:38:352
24808992020,0cyclictest0-21swapper/219:33:312
24808992020,0cyclictest0-21swapper/200:33:362
24808992018,2cyclictest2333-21idleruntime-cro19:30:002
24808992018,1cyclictest28109-21idleruntime-cro21:30:002
24808992018,1cyclictest10896-21sh21:59:592
2480899200,20cyclictest29178-21apt-get23:43:322
2480899200,20cyclictest0-21swapper/219:13:322
2480899200,0cyclictest18476-21apt-get00:28:342
2480899200,0cyclictest0-21swapper/219:18:332
2480899200,0cyclictest0-21swapper/200:08:332
24802992020,0cyclictest7090-21apt-get22:58:331
24802992020,0cyclictest7090-21apt-get22:58:331
24802992020,0cyclictest26245-21apt-get21:28:331
24802992020,0cyclictest0-21swapper/122:28:321
24802992020,0cyclictest0-21swapper/120:13:321
24802992020,0cyclictest0-21swapper/119:28:321
24802992020,0cyclictest0-21swapper/100:08:331
24802992019,1cyclictest27154-21perf19:15:011
24802992018,1cyclictest18545-21awk22:15:001
2480299200,20cyclictest0-21swapper/123:10:001
2480299200,20cyclictest0-21swapper/122:18:331
2480299200,20cyclictest0-21swapper/121:45:021
2480299200,20cyclictest0-21swapper/119:53:321
2480299200,0cyclictest7695-21apt-get19:43:341
2480299200,0cyclictest0-21swapper/122:40:011
2480299200,0cyclictest0-21swapper/119:23:321
24797992020,0cyclictest0-21swapper/023:43:320
24797992020,0cyclictest0-21swapper/023:18:330
24797992020,0cyclictest0-21swapper/023:13:320
24797992020,0cyclictest0-21swapper/023:00:020
24797992020,0cyclictest0-21swapper/022:18:330
24797992020,0cyclictest0-21swapper/021:18:330
24797992020,0cyclictest0-21swapper/000:33:360
24797992019,1cyclictest596-21in:imuxsock21:45:010
24797992019,1cyclictest26970-21apt-get20:23:330
24797992019,0cyclictest14884-21apt-get19:58:310
2479799200,20cyclictest0-21swapper/023:48:320
2479799200,19cyclictest0-21swapper/023:23:320
2479799200,0cyclictest0-21swapper/021:33:310
24809991918,0cyclictest19752-21apt-get20:08:323
24809991918,0cyclictest18632-21apt-get21:13:363
24809991918,0cyclictest0-21swapper/321:08:333
24809991917,1cyclictest4735-21apt19:34:593
2480999190,4cyclictest0-21swapper/321:28:333
24808991919,0cyclictest3844-21apt-get21:48:332
24808991919,0cyclictest22146-21apt-get20:13:312
24808991919,0cyclictest0-21swapper/223:53:332
24808991919,0cyclictest0-21swapper/223:28:342
24808991919,0cyclictest0-21swapper/221:43:332
24808991919,0cyclictest0-21swapper/221:08:342
24808991918,1cyclictest14240-21apt-get22:08:322
2480899190,0cyclictest0-21swapper/219:43:332
24802991919,0cyclictest17090-21apt-get23:18:331
24802991919,0cyclictest1040-21apt-get21:43:321
24802991919,0cyclictest0-21swapper/123:23:331
24802991919,0cyclictest0-21swapper/122:08:311
24802991919,0cyclictest0-21swapper/120:48:341
24802991918,1cyclictest766-21snmpd22:25:001
24802991918,1cyclictest2877-21apt-get19:33:311
2480299190,0cyclictest0-21swapper/123:13:331
24797991919,0cyclictest0-21swapper/021:38:350
24797991919,0cyclictest0-21swapper/020:38:330
24797991919,0cyclictest0-21swapper/020:38:330
24797991918,0cyclictest0-21swapper/023:33:330
24797991916,2cyclictest313-21systemd-journal23:55:000
24809991818,0cyclictest0-21swapper/321:45:023
24809991818,0cyclictest0-21swapper/320:43:353
24809991818,0cyclictest0-21swapper/320:13:323
24809991816,1cyclictest598-21rs:main3
24809991816,1cyclictest20332-21apt00:30:003
24809991816,1cyclictest0-21swapper/321:55:003
24808991818,0cyclictest0-21swapper/223:48:322
24808991818,0cyclictest0-21swapper/223:18:332
24808991818,0cyclictest0-21swapper/223:08:322
24808991818,0cyclictest0-21swapper/220:18:312
24808991818,0cyclictest0-21swapper/220:00:002
24808991816,1cyclictest28709-21apt-get22:38:342
24808991815,2cyclictest13717-21wc22:05:002
24802991818,0cyclictest0-21swapper/120:23:331
24802991818,0cyclictest0-21swapper/120:03:341
24802991818,0cyclictest0-21swapper/100:33:361
24797991818,0cyclictest24019-21perf20:15:000
24797991818,0cyclictest0-21swapper/022:08:310
24797991818,0cyclictest0-21swapper/021:08:340
24797991818,0cyclictest0-21swapper/000:18:320
24797991817,1cyclictest26435-21idleruntime-cro20:20:000
24797991816,1cyclictest0-21swapper/000:00:000
24809991717,0cyclictest0-21swapper/320:38:333
24809991717,0cyclictest0-21swapper/320:38:333
24809991716,1cyclictest31578-21apt-get23:48:333
24809991714,3cyclictest0-21swapper/322:28:323
24808991717,0cyclictest28645-21apt-get21:33:322
24808991717,0cyclictest0-21swapper/223:38:382
24808991717,0cyclictest0-21swapper/223:33:332
24808991717,0cyclictest0-21swapper/221:05:002
24808991717,0cyclictest0-21swapper/220:33:322
24808991716,1cyclictest0-21swapper/220:44:592
24802991717,0cyclictest31578-21apt-get23:48:321
24802991717,0cyclictest0-21swapper/123:43:321
24802991717,0cyclictest0-21swapper/123:28:331
24802991717,0cyclictest0-21swapper/122:38:331
24802991716,1cyclictest22-21ksoftirqd/100:03:341
2480299170,16cyclictest16785-21grep21:08:361
24797991717,0cyclictest974-21apt-get22:45:010
24797991717,0cyclictest0-21swapper/021:23:330
24797991717,0cyclictest0-21swapper/020:33:300
24797991716,0cyclictest0-21swapper/022:53:330
24797991714,2cyclictest313-21systemd-journal00:15:000
24809991616,0cyclictest0-21swapper/322:23:333
2480999160,5cyclictest0-21swapper/300:03:383
2480999160,15cyclictest15639-21sh21:05:003
2480999160,0cyclictest0-21swapper/320:53:333
24808991616,0cyclictest1750-21apt-get20:38:332
24808991616,0cyclictest1750-21apt-get20:38:332
24808991616,0cyclictest0-21swapper/223:13:332
24808991616,0cyclictest0-21swapper/221:23:332
24808991616,0cyclictest0-21swapper/219:38:312
24808991615,1cyclictest30489-21apt-get19:23:312
2480899160,2cyclictest10845-21perf20:55:002
24802991616,0cyclictest6242-21apt-get21:53:341
24802991616,0cyclictest18632-21apt-get21:13:361
24802991616,0cyclictest0-21swapper/123:33:331
24802991616,0cyclictest0-21swapper/120:08:331
24802991614,2cyclictest22-21ksoftirqd/121:00:001
24802991614,1cyclictest0-21swapper/122:45:001
24797991616,0cyclictest6474-21apt-get00:03:340
24797991616,0cyclictest0-21swapper/000:23:340
2479799160,15cyclictest6744-21cpuspeed_turbos20:48:350
2480899157,8cyclictest766-21snmpd20:00:202
24808991515,0cyclictest10092-21apt-get19:48:332
24808991512,2cyclictest313-21systemd-journal20:30:002
2480899150,14cyclictest16687-21expr00:23:362
24802991515,0cyclictest0-21swapper/122:33:331
2480299150,0cyclictest1561-21apt-get23:53:331
24797991515,0cyclictest0-21swapper/022:38:330
24797991512,2cyclictest28165-21idleruntime-cro22:34:590
2479799150,14cyclictest0-21swapper/023:38:430
2479799150,14cyclictest0-21swapper/022:20:390
24809991414,0cyclictest0-21swapper/322:50:273
24809991414,0cyclictest0-21swapper/320:26:193
2480999140,14cyclictest0-21swapper/323:20:263
2480999140,14cyclictest0-21swapper/322:30:223
2480999140,14cyclictest0-21swapper/321:30:153
2480999140,14cyclictest0-21swapper/320:20:273
2480999140,0cyclictest0-21swapper/322:07:323
2480899140,14cyclictest0-21swapper/222:50:482
2480899140,14cyclictest0-21swapper/221:11:292
2480899140,14cyclictest0-21swapper/219:50:562
2480899140,14cyclictest0-21swapper/200:35:482
2480899140,14cyclictest0-21swapper/200:35:482
2480299142,8cyclictest0-21swapper/122:55:001
24802991414,0cyclictest0-21swapper/121:30:401
24802991414,0cyclictest0-21swapper/120:50:241
2480299140,0cyclictest0-21swapper/123:01:491
2480299140,0cyclictest0-21swapper/121:16:171
2480299140,0cyclictest0-21swapper/100:28:081
24797991414,0cyclictest0-21swapper/019:45:370
24797991414,0cyclictest0-21swapper/019:30:060
2479799140,14cyclictest0-21swapper/022:10:060
2479799140,14cyclictest0-21swapper/021:01:440
2479799140,0cyclictest0-21swapper/020:05:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional