You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 23:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Apr 01, 2026 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
176325995443,5cyclictest0-21swapper/207:50:282
176323995043,4cyclictest0-21swapper/010:20:390
176325994931,14cyclictest358399-21aten_r2power_cu12:10:182
176324994940,5cyclictest1-21systemd10:15:011
176324994739,5cyclictest212294-21diskmemload12:20:011
176325994618,19cyclictest1-21systemd11:41:592
176324994638,5cyclictest374105-21cut12:30:011
176323994639,4cyclictest367474-21mailstats12:20:390
17632399462,36cyclictest221639-21ssh09:20:290
176324994536,6cyclictest318803-21ntpq11:20:281
176324994536,6cyclictest179347-21systemd-run07:20:011
176324994536,6cyclictest179347-21systemd-run07:20:011
176325994436,5cyclictest203294-21munin-run08:40:012
176325994427,14cyclictest209293-21awk09:00:012
176324994436,5cyclictest269522-21munin-plugin-st10:20:001
176324994436,5cyclictest192815-21cron08:05:001
17632399443,34cyclictest306760-21ssh11:05:300
176323994429,9cyclictest12350irq/32-eth010:05:290
176323994429,9cyclictest12350irq/32-eth010:05:290
176323994429,7cyclictest12350irq/32-eth011:15:310
176323994414,23cyclictest343033-21ssh11:50:320
176323994414,23cyclictest343033-21ssh11:50:320
17632599434,35cyclictest250205-21ssh09:55:302
176325994336,4cyclictest285499-21systemctl10:40:012
176325994333,7cyclictest234432-21unixbench_multi09:35:412
17632599432,36cyclictest211763-21ntpq09:05:332
17632599432,35cyclictest330878-21ntpq11:35:292
17632599432,33cyclictest262507-21ssh10:10:322
17632599431,35cyclictest178693-21ntpq07:15:292
17632599431,35cyclictest178693-21ntpq07:15:292
176325994310,17cyclictest212294-21diskmemload09:23:202
17632499434,32cyclictest298733-21ssh10:55:321
176324994334,6cyclictest421-21dbus-daemon12:05:021
176323994330,8cyclictest12450irq/33-eth012:35:310
17632599428,24cyclictest428-21mta-sts-daemon12:15:312
176325994233,5cyclictest421-21dbus-daemon12:05:382
176325994233,5cyclictest220512-21munin-plugin-st09:20:002
17632599422,35cyclictest286739-21ntpq10:40:342
17632599422,34cyclictest195186-21ntpq08:10:282
17632599422,33cyclictest210160-21ntpq09:00:282
17632599421,37cyclictest187799-21ntpq07:45:332
17632599421,36cyclictest293947-21/usr/sbin/munin10:50:202
17632599420,39cyclictest307602-21ssh11:06:012
17632399421,33cyclictest310918-21ssh11:10:330
17632399421,33cyclictest310918-21ssh11:10:330
17632599417,30cyclictest278064-21ssh10:30:152
17632599414,28cyclictest229873-21ssh09:30:322
176325994129,7cyclictest420-21avahi-daemon11:13:302
176325994129,7cyclictest420-21avahi-daemon11:13:302
17632599412,33cyclictest190647-21ntpq07:55:272
17632599411,35cyclictest274835-21ntpq10:25:342
17632599411,35cyclictest266690-21ntpq10:15:332
17632599411,35cyclictest253444-21ssh10:00:052
17632599411,35cyclictest192158-21/usr/sbin/munin08:00:272
17632599411,34cyclictest366390-21ssh12:20:132
17632599411,34cyclictest181729-21ntpq07:25:302
17632599411,33cyclictest243165-21ssh09:46:342
17632599411,33cyclictest243165-21ssh09:46:342
17632599411,23cyclictest205661-21ntpq08:45:292
176324994134,4cyclictest198052-21kworker/u8:3-rpciod09:30:181
176324994133,5cyclictest253217-21ssh10:00:011
176324994133,5cyclictest182329-21systemd-run07:30:001
17632399413,34cyclictest254320-21ssh10:00:300
17632599407,29cyclictest197961-21sshd08:20:252
176325994032,5cyclictest378189-21latency_hist12:35:022
17632599402,34cyclictest302747-21ntpq11:00:292
17632599401,36cyclictest320710-21ssh11:23:002
17632599401,35cyclictest340379-21ssh11:47:052
17632599401,35cyclictest227128-21ssh09:27:062
17632599401,35cyclictest227128-21ssh09:27:062
17632599401,34cyclictest379268-21ntpq12:35:312
17632599401,34cyclictest313986-21ssh11:15:092
17632599401,34cyclictest204304-21/usr/sbin/munin08:40:322
17632599401,34cyclictest180309-21ntpq07:20:342
176324994032,3cyclictest0-21swapper/109:15:291
176324994031,6cyclictest241116-21latency_hist09:45:031
176324994027,10cyclictest282445-21missed_timers10:35:261
17632599393,32cyclictest421-21dbus-daemon11:58:382
176325993914,20cyclictest0-21swapper/212:25:242
17632599391,36cyclictest301083-21ssh10:59:002
17632599391,36cyclictest259470-21ssh10:06:272
17632599391,36cyclictest259470-21ssh10:06:272
17632599391,34cyclictest421-21dbus-daemon08:09:332
17632599391,34cyclictest421-21dbus-daemon08:09:332
17632599391,34cyclictest326076-21ssh11:30:112
17632599391,34cyclictest272465-21ssh10:22:492
17632599391,34cyclictest199694-21ntpq08:25:302
17632599391,33cyclictest350284-21ssh12:00:132
17632599391,33cyclictest207202-21ntpq08:50:292
176324993931,5cyclictest249060-21cron09:55:001
176324993930,6cyclictest261-21systemd-journal12:40:011
176324993930,5cyclictest206826-21fschecks_time08:50:221
176324993929,6cyclictest279599-21sshd10:31:251
17632499392,31cyclictest233952-21ssh09:35:321
176323993933,4cyclictest289541-21munin-plugin-st10:45:010
176323993931,6cyclictest374105-21munin-plugin-st12:30:010
176323993931,5cyclictest420-21avahi-daemon09:26:290
176323993931,5cyclictest420-21avahi-daemon09:26:290
17632599381,35cyclictest344080-21ssh11:51:302
17632599381,35cyclictest344080-21ssh11:51:302
17632599381,34cyclictest323876-21ssh11:26:212
17632599381,34cyclictest245290-21ssh09:50:112
17632599381,34cyclictest214512-21ssh09:11:272
17632599381,33cyclictest238008-21ntpq09:40:322
17632599381,33cyclictest176537-21/usr/sbin/munin07:10:432
17632599381,33cyclictest176537-21/usr/sbin/munin07:10:432
17632599381,32cyclictest186211-21ntpq07:40:282
176325993811,23cyclictest290557-21needreboot10:45:272
176324993832,3cyclictest0-21swapper/110:47:291
176324993830,5cyclictest270114-21aten_r2power_vo10:20:231
176324993830,5cyclictest180820-21sh07:25:001
176324993827,8cyclictest261-21systemd-journal11:40:311
176323993825,11cyclictest176322-21cyclictest08:45:380
17632399382,30cyclictest282638-21ssh10:35:320
17632599371,33cyclictest200886-21/usr/sbin/munin08:30:242
17632599371,33cyclictest183256-21ntpq07:30:322
176324993731,3cyclictest0-21swapper/109:00:011
176324993727,6cyclictest362023-21munin-run12:15:011
176324993726,8cyclictest287107-21timerwakeupswit10:40:411
176323993730,3cyclictest0-21swapper/009:40:010
176323993727,7cyclictest303212-21timerandwakeup11:00:400
176323993723,8cyclictest262537-21sh10:10:330
17632599361,32cyclictest196425-21/usr/sbin/munin08:15:242
17632599361,32cyclictest184708-21ntpq07:35:312
176324993627,6cyclictest421-21dbus-daemon09:45:231
176324993627,6cyclictest421-21dbus-daemon09:45:231
176323993629,4cyclictest198052-21kworker/u8:3-xprtiod08:40:010
176323993628,3cyclictest0-21swapper/007:30:290
176323993614,14cyclictest151ktimers/008:20:310
17632499356,22cyclictest347064-21ssh11:55:321
176324993530,2cyclictest0-21swapper/107:40:011
176324993528,3cyclictest0-21swapper/112:30:331
176324993527,5cyclictest259058-21head10:05:431
176324993527,5cyclictest259058-21head10:05:431
176324993526,6cyclictest0-21swapper/112:10:011
176324993526,4cyclictest0-21swapper/111:50:341
176324993526,4cyclictest0-21swapper/111:50:341
176324993523,9cyclictest222202-21timerwakeupswit09:20:411
17632399353,25cyclictest351276-21ssh12:00:360
176323993529,3cyclictest0-21swapper/010:55:310
176323993527,5cyclictest420-21avahi-daemon09:56:010
176323993524,8cyclictest346837-21latency11:55:270
17632399352,30cyclictest242037-21ssh09:45:300
17632399352,30cyclictest242037-21ssh09:45:300
176324993427,4cyclictest1051-21mta-sts-daemon07:45:311
176324993427,3cyclictest0-21swapper/111:40:011
176324993426,5cyclictest295209-21timerwakeupswit10:50:411
17632399342,27cyclictest338955-21ssh11:45:290
176324993326,4cyclictest0-21swapper/107:55:321
176324993326,3cyclictest0-21swapper/111:03:141
176324993325,5cyclictest339444-21timerandwakeup11:45:421
176324993325,5cyclictest306730-21ntp_kernel_err11:05:301
176324993325,5cyclictest0-21swapper/109:05:341
176324993325,3cyclictest0-21swapper/111:10:321
176324993325,3cyclictest0-21swapper/111:10:321
176324993324,4cyclictest0-21swapper/109:13:261
176324993323,4cyclictest0-21swapper/109:25:301
176324993323,4cyclictest0-21swapper/109:25:301
176323993328,4cyclictest176322-21cyclictest10:49:200
176323993328,2cyclictest0-21swapper/010:15:400
176323993327,4cyclictest439-21systemd-logind08:00:010
176323993327,4cyclictest277746-21ssh10:30:010
176323993325,5cyclictest216455-21latency_hist09:15:020
176323993324,7cyclictest378104-21cron12:35:010
176323993323,8cyclictest151ktimers/009:40:350
176323993323,7cyclictest197472-21latency_hist08:20:020
176323993320,8cyclictest12350irq/32-eth011:25:340
176323993311,15cyclictest12350irq/32-eth011:30:300
17632599320,29cyclictest176336-21munin-run07:10:012
176324993225,4cyclictest274520-21latency10:25:271
176324993225,4cyclictest253895-21ssh10:00:231
176324993225,4cyclictest177797-21cron07:15:001
176324993225,4cyclictest177797-21cron07:15:001
176324993224,4cyclictest327084-21proc_pri11:30:371
176323993226,4cyclictest1051-21mta-sts-daemon10:54:380
176323993220,8cyclictest151ktimers/007:40:330
176323993216,10cyclictest12350irq/32-eth009:30:320
176324993124,4cyclictest314392-21ssh11:15:221
176324993124,4cyclictest1051-21mta-sts-daemon11:27:001
176323993125,4cyclictest355072-21ntp_kernel_pll_12:05:290
176323993122,6cyclictest334916-21ntp_kernel_err11:40:280
176323993119,8cyclictest3487805-21cupsd11:21:560
176323993115,12cyclictest151ktimers/009:05:280
17632399311,25cyclictest0-21swapper/011:35:330
176324993022,5cyclictest194325-21systemd-run08:10:011
176324993022,5cyclictest194325-21systemd-run08:10:011
176323993022,5cyclictest245590-21cpu09:50:200
17632399302,22cyclictest217558-21sh09:15:300
176323993012,13cyclictest151ktimers/008:25:310
176324992923,3cyclictest0-21swapper/112:22:341
176324992923,3cyclictest0-21swapper/108:45:011
176324992922,4cyclictest200151-21unixbench-2d08:25:401
176324992921,5cyclictest197554-21acpi08:20:191
176324992920,6cyclictest420-21avahi-daemon08:45:371
176323992924,3cyclictest279104-21mailstats10:30:370
176324992822,3cyclictest0-21swapper/109:00:361
176324992820,4cyclictest0-21swapper/107:45:021
176323992822,4cyclictest179842-21fschecks_count07:20:220
176323992820,6cyclictest209284-21cat09:00:010
176323992816,6cyclictest358544-21aten_r2power_vo12:10:200
176323992720,3cyclictest0-21swapper/012:19:560
176324992618,5cyclictest196336-21gltestperf08:15:231
17632399262,19cyclictest3487805-21cupsd08:00:330
176323992621,3cyclictest207785-21cron08:55:010
17632399261,20cyclictest659-21snmpd07:45:320
176324992519,3cyclictest0-21swapper/107:55:011
176324992513,9cyclictest421-21dbus-daemon07:35:011
176323992519,3cyclictest1056-21mta-sts-daemon08:40:250
176323992517,6cyclictest189808-21sh07:55:010
17632499249,12cyclictest271ktimers/108:35:341
176324992417,4cyclictest1055-21mta-sts-daemon08:10:381
176324992414,6cyclictest176335-21systemd-run07:10:011
176323992417,4cyclictest194458-21latency_hist08:10:020
176323992417,4cyclictest194458-21latency_hist08:10:020
176323992413,9cyclictest0-21swapper/007:15:400
176323992413,9cyclictest0-21swapper/007:15:400
17632399238,13cyclictest421-21dbus-daemon08:35:010
17632399232,15cyclictest195194-21ntpq08:10:280
176323992317,4cyclictest184622-21needreboot07:35:280
176323992315,6cyclictest210772-21sshd09:01:380
17632399231,18cyclictest441-21gpm07:25:280
176323992213,7cyclictest177754-21unixbench_singl07:10:410
176323992213,7cyclictest177754-21unixbench_singl07:10:410
17632499211,6cyclictest200950-21kernelversion08:30:251
17632699202,10cyclictest0-21swapper/309:15:353
17632699181,10cyclictest0-21swapper/309:55:323
176323991813,3cyclictest1051-21mta-sts-daemon07:10:010
17632699172,7cyclictest0-21swapper/310:25:313
17632699171,8cyclictest0-21swapper/312:30:293
17632699163,8cyclictest0-21swapper/309:10:333
17632699163,7cyclictest0-21swapper/312:10:293
17632699162,8cyclictest0-21swapper/310:55:313
17632699155,7cyclictest0-21swapper/311:05:313
17632699153,8cyclictest0-21swapper/311:50:313
17632699153,8cyclictest0-21swapper/311:50:313
17632699153,5cyclictest0-21swapper/309:45:313
17632699153,5cyclictest0-21swapper/309:45:313
17632699152,8cyclictest0-21swapper/312:00:333
17632699152,8cyclictest0-21swapper/311:49:173
17632699152,8cyclictest0-21swapper/311:35:303
17632699152,8cyclictest0-21swapper/309:50:333
17632699152,8cyclictest0-21swapper/309:35:333
17632699152,8cyclictest0-21swapper/308:20:003
17632699152,7cyclictest0-21swapper/311:10:293
17632699152,7cyclictest0-21swapper/311:10:293
17632699152,7cyclictest0-21swapper/310:20:293
17632699152,7cyclictest0-21swapper/309:40:293
17632699152,6cyclictest0-21swapper/312:15:273
17632699151,9cyclictest0-21swapper/310:40:283
17632699151,8cyclictest0-21swapper/309:20:303
17632699151,6cyclictest0-21swapper/307:15:293
17632699151,6cyclictest0-21swapper/307:15:293
17632699143,8cyclictest0-21swapper/310:35:283
17632699143,7cyclictest0-21swapper/311:30:323
17632699143,7cyclictest0-21swapper/311:00:323
17632699143,7cyclictest0-21swapper/309:30:363
17632699142,7cyclictest0-21swapper/307:25:013
17632699142,6cyclictest0-21swapper/307:35:023
17632699141,8cyclictest0-21swapper/312:20:363
17632699141,8cyclictest0-21swapper/310:30:333
17632699141,8cyclictest0-21swapper/309:25:353
17632699141,8cyclictest0-21swapper/309:25:353
17632699141,7cyclictest0-21swapper/312:25:253
17632699141,7cyclictest0-21swapper/310:15:263
17632699141,7cyclictest0-21swapper/310:05:313
17632699141,7cyclictest0-21swapper/310:05:313
17632699141,7cyclictest0-21swapper/308:10:283
17632699141,7cyclictest0-21swapper/307:35:313
17632699141,6cyclictest0-21swapper/308:00:303
17632699132,8cyclictest0-21swapper/310:45:303
17632699132,7cyclictest0-21swapper/311:20:293
17632699132,7cyclictest0-21swapper/310:10:413
17632699132,6cyclictest0-21swapper/312:35:233
17632699132,6cyclictest0-21swapper/311:25:253
17632699132,6cyclictest0-21swapper/308:45:303
17632699132,5cyclictest0-21swapper/307:45:283
17632699131,7cyclictest0-21swapper/311:40:313
17632699131,7cyclictest0-21swapper/310:00:303
17632699131,7cyclictest0-21swapper/309:10:023
17632699131,7cyclictest0-21swapper/308:50:303
17632699131,6cyclictest0-21swapper/312:05:193
17632699131,6cyclictest0-21swapper/308:05:293
17632699131,6cyclictest0-21swapper/308:05:293
17632699131,6cyclictest0-21swapper/307:55:303
17632699131,6cyclictest0-21swapper/307:25:273
17632699123,4cyclictest0-21swapper/307:50:273
17632699121,8cyclictest0-21swapper/311:55:263
17632699121,7cyclictest0-21swapper/311:15:273
17632699121,7cyclictest0-21swapper/310:50:283
17632699121,6cyclictest0-21swapper/308:55:283
17632699112,5cyclictest0-21swapper/308:35:263
17632699112,5cyclictest0-21swapper/307:10:403
17632699112,5cyclictest0-21swapper/307:10:403
17632699111,5cyclictest0-21swapper/309:00:233
17632699111,5cyclictest0-21swapper/307:40:263
17632699102,5cyclictest0-21swapper/308:40:263
17632699101,5cyclictest0-21swapper/308:30:283
17632699101,5cyclictest0-21swapper/308:25:303
1763269990,5cyclictest0-21swapper/308:20:193
1763269981,3cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional