You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 20:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Sun Dec 07, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
50299290,20cyclictest0-21swapper/212:13:292
50299252,22cyclictest16077-21cron10:59:592
50299240,23cyclictest5720-21idleruntime-cro07:20:002
50599230,2cyclictest21588-21dump-pmu-power09:00:003
50599230,22cyclictest4688-21sh08:25:003
50599230,22cyclictest28882-21munin-run10:20:003
50599230,22cyclictest14003-21latency_hist09:50:013
502992321,1cyclictest598-21rs:main2
50299230,1cyclictest8769-21apt-get11:50:012
49999230,22cyclictest19271-21perf10:00:001
49999230,22cyclictest0-21swapper/110:30:001
494992322,1cyclictest3529-21apt-get07:15:020
49499230,22cyclictest21559-21latency_hist09:00:000
505992221,1cyclictest13686-21perf10:55:003
505992220,2cyclictest34-21ksoftirqd/310:14:593
505992220,2cyclictest34-21ksoftirqd/308:45:003
505992220,1cyclictest29893-21dump-pmu-power08:10:003
505992220,1cyclictest11614-21cat09:45:003
505992220,1cyclictest0-21swapper/312:00:023
505992219,2cyclictest34-21ksoftirqd/311:50:003
505992219,2cyclictest34-21ksoftirqd/311:45:003
50599220,21cyclictest593-21dbus-daemon12:15:013
50599220,21cyclictest17857-21basename07:44:593
50599220,21cyclictest0-21swapper/312:23:323
50599220,21cyclictest0-21swapper/311:50:013
50599220,21cyclictest0-21swapper/311:33:303
50599220,21cyclictest0-21swapper/308:13:283
50599220,21cyclictest0-21swapper/307:48:303
50599220,1cyclictest9200-21sendmail-msp09:40:003
502992222,0cyclictest0-21swapper/210:08:292
502992220,1cyclictest612-21gdbus08:20:002
502992220,1cyclictest0-21swapper/208:54:592
50299220,21cyclictest0-21swapper/208:13:292
50299220,21cyclictest0-21swapper/208:10:002
499992220,1cyclictest13351-21wc12:00:001
499992220,1cyclictest0-21swapper/112:08:301
49999220,22cyclictest0-21swapper/109:38:281
49999220,21cyclictest5364-21awk11:45:001
49999220,21cyclictest4920-21apt-get09:33:291
49999220,21cyclictest26983-21apt-get10:18:311
49999220,21cyclictest23408-21sendmail12:20:011
49999220,1cyclictest0-21swapper/112:39:591
494992222,0cyclictest21573-21apt-get09:00:020
494992221,0cyclictest0-21swapper/010:03:320
494992220,1cyclictest29169-21cron09:15:000
494992220,1cyclictest16386-21awk09:55:000
494992220,1cyclictest13982-21apt-get12:00:020
494992220,1cyclictest11929-21perf08:40:000
494992220,1cyclictest0-21swapper/007:25:010
49499220,21cyclictest19256-21apt10:00:000
49499220,1cyclictest7317-21apt-get09:38:290
505992121,0cyclictest0-21swapper/312:28:293
505992120,1cyclictest28677-21apt-get11:28:323
505992120,1cyclictest0-21swapper/312:33:323
505992119,1cyclictest0-21swapper/310:00:003
50599210,20cyclictest0-21swapper/309:08:293
50599210,20cyclictest0-21swapper/308:03:293
50599210,0cyclictest0-21swapper/311:13:293
502992120,1cyclictest26342-21apt-get12:28:292
502992120,1cyclictest21528-21apt-get12:18:292
502992120,0cyclictest28746-21apt-get12:33:322
502992120,0cyclictest20819-21apt-get07:53:302
502992120,0cyclictest19068-21apt-get11:08:312
502992119,1cyclictest13687-21munin-run10:55:002
50299210,20cyclictest18423-21apt-get07:48:302
50299210,20cyclictest11603-21latency_hist09:45:002
50299210,20cyclictest0-21swapper/210:38:322
50299210,20cyclictest0-21swapper/209:28:292
50299210,1cyclictest12131-21apt-get09:48:292
50299210,0cyclictest604-21grep12:40:002
50299210,0cyclictest21461-21apt-get11:13:302
50299210,0cyclictest16384-21apt-get09:55:002
499992120,0cyclictest0-21swapper/109:08:301
499992120,0cyclictest0-21swapper/108:48:291
499992119,1cyclictest31951-21cron09:20:001
49999210,21cyclictest14283-21apt-get12:03:311
49999210,21cyclictest0-21swapper/111:33:301
49999210,21cyclictest0-21swapper/108:38:291
49999210,20cyclictest14201-21apt-get10:58:321
49999210,0cyclictest2916-21apt-get07:15:021
49499219,11cyclictest4394-21perf09:29:590
494992121,0cyclictest0-21swapper/011:43:310
494992120,1cyclictest2841-21apt-get08:23:290
494992120,1cyclictest0-21swapper/012:33:310
494992120,0cyclictest0-21swapper/011:13:290
494992119,1cyclictest579-21perf12:40:000
494992119,1cyclictest29178-21apt-get09:15:010
494992119,1cyclictest12910-21cron07:35:000
49499210,20cyclictest0-21swapper/011:18:290
49499210,20cyclictest0-21swapper/010:53:310
49499210,0cyclictest0-21swapper/009:33:280
50599205,9cyclictest151rcu_preempt10:43:293
505992020,0cyclictest0-21swapper/311:08:303
505992020,0cyclictest0-21swapper/310:48:283
505992020,0cyclictest0-21swapper/310:08:293
505992020,0cyclictest0-21swapper/307:13:313
505992019,0cyclictest0-21swapper/311:58:313
505992018,1cyclictest0-21swapper/309:15:003
50599200,0cyclictest32523-21apt-get09:23:303
50599200,0cyclictest1059-21apt-get11:38:313
50599200,0cyclictest0-21swapper/310:38:323
50599200,0cyclictest0-21swapper/308:53:333
502992020,0cyclictest3488-21apt-get11:43:302
502992020,0cyclictest25631-21apt-get08:03:292
502992020,0cyclictest0-21swapper/212:23:322
502992020,0cyclictest0-21swapper/212:03:312
502992020,0cyclictest0-21swapper/210:48:292
502992020,0cyclictest0-21swapper/209:00:022
502992019,0cyclictest0-21swapper/211:28:322
502992018,1cyclictest9219-21munin-run09:40:002
502992018,1cyclictest3655-21awk10:35:002
50299200,20cyclictest0-21swapper/209:23:302
50299200,0cyclictest0-21swapper/208:58:322
499992020,0cyclictest23950-21apt-get12:23:321
499992020,0cyclictest0-21swapper/112:33:321
499992020,0cyclictest0-21swapper/112:13:291
499992020,0cyclictest0-21swapper/111:38:301
499992020,0cyclictest0-21swapper/111:23:311
499992020,0cyclictest0-21swapper/111:03:281
499992020,0cyclictest0-21swapper/109:53:311
499992020,0cyclictest0-21swapper/109:28:291
499992020,0cyclictest0-21swapper/107:53:301
499992020,0cyclictest0-21swapper/107:33:281
499992020,0cyclictest0-21swapper/107:13:301
499992019,0cyclictest0-21swapper/112:28:291
499992018,1cyclictest9516-21basename08:34:591
49999200,19cyclictest0-21swapper/107:43:291
49999200,0cyclictest0-21swapper/110:38:321
49999200,0cyclictest0-21swapper/110:13:271
494992020,0cyclictest28028-21apt-get08:08:290
494992020,0cyclictest0-21swapper/011:38:300
494992020,0cyclictest0-21swapper/010:33:290
494992020,0cyclictest0-21swapper/010:18:310
494992020,0cyclictest0-21swapper/008:28:290
494992020,0cyclictest0-21swapper/008:18:280
494992020,0cyclictest0-21swapper/007:48:300
49499200,0cyclictest0-21swapper/012:18:300
49499200,0cyclictest0-21swapper/011:51:440
49499200,0cyclictest0-21swapper/011:45:010
50599198,10cyclictest331rcuc/309:33:293
505991919,0cyclictest0-21swapper/309:28:293
505991919,0cyclictest0-21swapper/308:38:293
505991919,0cyclictest0-21swapper/307:38:293
505991918,1cyclictest34-21ksoftirqd/312:40:003
505991918,1cyclictest34-21ksoftirqd/311:00:003
502991919,0cyclictest6282-21apt-get11:48:302
502991919,0cyclictest31070-21apt-get11:33:302
502991919,0cyclictest0-21swapper/210:18:312
502991919,0cyclictest0-21swapper/207:23:292
502991917,1cyclictest16401-21perf09:54:592
502991917,1cyclictest15447-21latency_hist07:40:002
50299190,18cyclictest0-21swapper/207:28:282
499991919,0cyclictest32523-21apt-get09:23:301
499991919,0cyclictest19704-21apt-get08:58:321
499991919,0cyclictest0-21swapper/110:33:301
499991919,0cyclictest0-21swapper/108:13:291
499991919,0cyclictest0-21swapper/107:48:311
499991918,1cyclictest0-21swapper/110:05:001
499991917,1cyclictest21460-21apt11:13:301
49999190,0cyclictest0-21swapper/107:23:291
494991919,0cyclictest26278-21apt-get11:23:290
494991919,0cyclictest24576-21apt-get10:13:270
494991919,0cyclictest0-21swapper/010:23:300
494991919,0cyclictest0-21swapper/009:23:300
494991919,0cyclictest0-21swapper/009:10:000
494991919,0cyclictest0-21swapper/008:30:310
494991918,1cyclictest0-21swapper/007:53:290
494991917,1cyclictest8492-21munin-run10:45:000
494991917,1cyclictest22185-21apt-get10:08:290
505991818,0cyclictest0-21swapper/311:23:313
505991818,0cyclictest0-21swapper/309:18:293
505991818,0cyclictest0-21swapper/309:03:273
505991818,0cyclictest0-21swapper/308:28:293
505991818,0cyclictest0-21swapper/307:53:303
505991817,1cyclictest34-21ksoftirqd/310:30:003
505991817,1cyclictest34-21ksoftirqd/310:25:003
505991817,1cyclictest0-21swapper/312:18:303
502991818,0cyclictest0-21swapper/210:13:272
502991818,0cyclictest0-21swapper/208:43:302
502991818,0cyclictest0-21swapper/208:38:292
499991818,0cyclictest5236-21apt-get08:28:291
499991818,0cyclictest0-21swapper/111:08:311
499991818,0cyclictest0-21swapper/110:08:291
499991818,0cyclictest0-21swapper/109:03:281
499991818,0cyclictest0-21swapper/107:58:291
499991818,0cyclictest0-21swapper/107:58:291
499991817,1cyclictest9737-21apt-get09:43:301
499991817,1cyclictest8161-21apt-get11:50:011
499991817,1cyclictest593-21dbus-daemon08:25:001
499991816,1cyclictest8167-21basename11:50:001
499991816,1cyclictest2328-21sendmail-msp08:20:001
49499189,7cyclictest0-21swapper/008:03:280
494991818,0cyclictest4195-21apt-get10:38:320
494991818,0cyclictest0-21swapper/011:08:310
494991818,0cyclictest0-21swapper/010:28:310
494991818,0cyclictest0-21swapper/009:48:290
494991818,0cyclictest0-21swapper/008:53:330
494991818,0cyclictest0-21swapper/007:58:290
494991818,0cyclictest0-21swapper/007:58:280
49499180,0cyclictest0-21swapper/008:48:290
505991717,0cyclictest1792-21apt-get10:33:303
505991717,0cyclictest0-21swapper/312:08:303
505991717,0cyclictest0-21swapper/307:28:303
505991715,1cyclictest0-21swapper/309:55:003
50599170,16cyclictest24503-21grep07:58:363
50599170,16cyclictest24503-21grep07:58:363
502991717,0cyclictest0-21swapper/209:08:302
502991717,0cyclictest0-21swapper/208:28:292
502991716,1cyclictest11049-21apt-get07:33:282
502991716,0cyclictest23214-21apt-get07:58:292
502991716,0cyclictest23214-21apt-get07:58:292
502991714,2cyclictest31271-21latency_hist10:25:002
499991717,0cyclictest0-21swapper/107:28:301
49999170,17cyclictest0-21swapper/108:03:291
494991717,0cyclictest9400-21apt-get10:48:280
494991717,0cyclictest19127-21apt-get12:13:290
494991717,0cyclictest0-21swapper/012:08:290
49499170,0cyclictest0-21swapper/009:43:300
50599163,5cyclictest151rcu_preempt07:33:383
505991616,0cyclictest0-21swapper/311:18:293
505991614,2cyclictest34-21ksoftirqd/307:15:013
505991614,1cyclictest0-21swapper/308:20:003
502991616,0cyclictest1031-21apt-get07:13:312
502991616,0cyclictest0-21swapper/211:38:312
502991616,0cyclictest0-21swapper/208:23:292
502991615,1cyclictest26278-21apt-get11:23:312
502991615,0cyclictest14850-21apt-get08:48:302
502991615,0cyclictest11469-21apt-get11:58:312
502991614,1cyclictest8484-21perf10:45:002
499991616,0cyclictest13446-21apt-get07:38:291
499991616,0cyclictest0-21swapper/109:48:281
499991616,0cyclictest0-21swapper/108:43:301
499991615,1cyclictest29397-21apt-get10:23:301
494991616,0cyclictest0-21swapper/011:58:310
494991616,0cyclictest0-21swapper/011:28:320
494991615,0cyclictest16614-21apt-get11:03:280
49499160,15cyclictest0-21swapper/010:58:360
49499160,15cyclictest0-21swapper/007:43:290
49499160,0cyclictest0-21swapper/012:23:320
50599159,5cyclictest34-21ksoftirqd/308:50:003
505991515,0cyclictest0-21swapper/311:03:283
505991515,0cyclictest0-21swapper/307:23:293
502991515,0cyclictest0-21swapper/211:03:282
502991515,0cyclictest0-21swapper/210:28:302
502991513,1cyclictest17751-21runrttasks07:44:162
502991513,1cyclictest0-21swapper/212:08:462
50299150,1cyclictest0-21swapper/208:33:322
50299150,14cyclictest0-21swapper/209:30:562
499991515,0cyclictest0-21swapper/111:28:321
494991513,1cyclictest0-21swapper/011:33:300
49499150,4cyclictest0-21swapper/007:38:290
49499150,15cyclictest766-21snmpd08:42:500
505991414,0cyclictest0-21swapper/310:02:163
50599140,0cyclictest0-21swapper/308:31:083
502991414,0cyclictest0-21swapper/210:00:562
50299140,14cyclictest0-21swapper/211:15:172
50299140,14cyclictest0-21swapper/209:10:492
499991414,0cyclictest0-21swapper/110:45:001
499991414,0cyclictest0-21swapper/110:40:131
499991414,0cyclictest0-21swapper/109:10:121
49999140,14cyclictest0-21swapper/111:15:061
49999140,14cyclictest0-21swapper/110:50:541
49999140,14cyclictest0-21swapper/108:50:061
494991414,0cyclictest0-21swapper/007:25:250
494991414,0cyclictest0-21swapper/007:10:440
49499140,14cyclictest0-21swapper/012:26:550
49499140,14cyclictest0-21swapper/008:10:010
49999130,0cyclictest0-21swapper/108:05:441
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional