You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 07:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue Apr 07, 2026 00:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3347728995950,6cyclictest439-21systemd-logind22:10:022
3347726995439,13cyclictest420-21avahi-daemon21:40:080
3347728994839,6cyclictest3489374-21systemd-run23:20:012
3347728994836,7cyclictest1457231-21cups-browsed22:35:322
3347726994842,4cyclictest3424315-21munin-run22:00:010
3347727994739,5cyclictest3404110-21cut21:35:011
3347727994737,6cyclictest3383714-21idleruntime-cro21:10:001
3347727994638,5cyclictest3446444-21grep22:25:441
3347727994638,5cyclictest3446444-21grep22:25:441
3347728994536,4cyclictest3452508-21kworker/u8:6-xprtiod22:45:212
3347727994537,5cyclictest3444661-21basename22:25:011
3347727994537,5cyclictest3444661-21basename22:25:011
3347728994436,5cyclictest3521737-21munin-plugin-st00:00:012
3347728994334,4cyclictest0-21swapper/223:10:352
3347728994333,6cyclictest0-21swapper/223:40:312
3347727994333,7cyclictest420-21avahi-daemon23:34:271
3347727994232,6cyclictest420-21avahi-daemon22:36:491
3347728994133,5cyclictest3436520-21grep22:15:012
3347728994133,5cyclictest3393683-21unixbench_multi21:20:462
3347728994132,6cyclictest3498459-21needreboot23:30:302
3347728994131,6cyclictest439-21systemd-logind22:25:022
3347728994131,6cyclictest439-21systemd-logind22:25:022
3347727994134,4cyclictest3554179-21readlink00:40:011
3347727994132,5cyclictest695-21cron19:45:001
3347727994132,5cyclictest695-21cron19:45:001
3347728994034,3cyclictest0-21swapper/222:00:012
3347728994032,5cyclictest3550118-21munin-run00:35:012
3347727994033,5cyclictest3424338-21tr22:00:011
3347727994031,6cyclictest421-21dbus-daemon23:22:541
3347727994031,5cyclictest3528980-21sshd00:08:201
3347728993930,6cyclictest3377708-21grep20:50:012
3347728993930,5cyclictest1051-21mta-sts-daemon21:45:362
3347728993926,9cyclictest1457231-21cups-browsed19:50:332
334772899391,22cyclictest3543279-21ssh00:25:362
3347726993932,4cyclictest3387807-21latency_hist21:15:010
3347726993931,6cyclictest3356709-21idleruntime-cro19:40:000
3347726993931,5cyclictest3396895-21latency21:25:280
3347728993830,4cyclictest0-21swapper/223:05:342
3347727993830,5cyclictest3355208-21grep19:35:011
3347727993828,8cyclictest3519048-21sh23:55:401
3347726993827,8cyclictest3513960-21/usr/sbin/munin23:50:240
3347728993730,4cyclictest3458565-21unixbench_multi22:40:432
3347727993729,5cyclictest3494354-21missed_timers23:25:291
3347727993728,6cyclictest1457231-21cups-browsed23:40:321
3347727993728,6cyclictest1457228-21cupsd22:40:071
3347726993731,3cyclictest0-21swapper/021:45:230
3347726993730,5cyclictest1457231-21cups-browsed21:31:310
3347726993730,5cyclictest1457231-21cups-browsed21:21:260
3347726993729,5cyclictest421-21dbus-daemon00:32:400
3347726993728,4cyclictest3440569-21munin-run22:20:010
3347728993628,5cyclictest1457231-21cups-browsed21:42:202
3347728993627,5cyclictest3355195-21cron19:35:012
3347727993631,3cyclictest0-21swapper/122:20:031
3347727993628,5cyclictest3477176-21munin-run23:05:011
3347727993628,5cyclictest3452772-21ssh22:35:011
3347727993628,5cyclictest3452772-21ssh22:35:011
3347727993628,5cyclictest3395988-21cron21:25:011
3347727993627,7cyclictest3517631-21munin-plugin-st23:55:011
3347727993627,5cyclictest420-21avahi-daemon21:27:401
3347727993625,8cyclictest421-21dbus-daemon19:55:301
3347727993625,8cyclictest3466145-21collect222:50:341
3347726993630,4cyclictest3448722-21systemd-run22:30:010
3347726993630,4cyclictest3448722-21systemd-run22:30:010
3347726993630,3cyclictest0-21swapper/000:25:010
3347726993628,5cyclictest3499427-21sh23:31:080
3347726993627,5cyclictest3383707-21diskmemload23:40:350
3347726993625,8cyclictest3522875-21sh00:00:320
3347728993529,3cyclictest0-21swapper/200:40:012
3347728993528,4cyclictest1056-21mta-sts-daemon00:20:292
3347728993527,5cyclictest3530055-21latency_hist00:10:032
3347728993527,5cyclictest3376212-21idleruntime-cro20:45:012
3347727993530,2cyclictest0-21swapper/123:50:021
3347727993529,3cyclictest0-21swapper/121:15:011
3347727993528,5cyclictest3361218-21systemd-run19:55:011
3347727993527,5cyclictest3426117-21ssh22:00:441
3347727993525,7cyclictest3534304-21/usr/sbin/munin00:15:431
3347726993530,3cyclictest3449355-21ls22:30:240
3347726993530,3cyclictest3449355-21ls22:30:240
3347726993529,4cyclictest3489318-21cron23:20:000
3347726993527,6cyclictest3519196-21timerandwakeup23:55:430
3347726993527,6cyclictest3471774-21sshd22:57:330
3347728993428,3cyclictest0-21swapper/223:25:012
3347728993427,3cyclictest0-21swapper/221:30:272
3347728993426,5cyclictest3534004-21latency_hist00:15:022
3347728993426,5cyclictest3534004-21latency_hist00:15:022
3347728993426,4cyclictest0-21swapper/221:15:322
3347728993420,11cyclictest3358217-21munin-plugin-st19:45:002
3347728993420,11cyclictest3358217-21munin-plugin-st19:45:002
3347727993427,3cyclictest0-21swapper/121:50:381
3347727993427,3cyclictest0-21swapper/121:45:271
3347727993426,5cyclictest3473075-21munin-run23:00:011
3347727993426,5cyclictest3355666-21cpuspeed19:35:231
3347727993424,7cyclictest3434051-21timerandwakeup22:10:441
3347727993423,8cyclictest695-21cron21:45:001
3347726993428,4cyclictest3453341-21aten_r2power_vo22:35:230
3347726993428,4cyclictest3420034-21kworker/u8:1+xprtiod22:05:010
3347726993427,3cyclictest0-21swapper/023:35:340
3347726993426,5cyclictest3521704-21cupsd00:19:050
3347726993424,7cyclictest3526920-21collect200:05:310
3347726993424,7cyclictest0-21swapper/020:30:320
3347728993329,2cyclictest0-21swapper/219:40:002
3347728993325,6cyclictest0-21swapper/223:25:342
3347728993324,6cyclictest516-21in:imuxsock23:40:002
3347728993324,6cyclictest3474476-21wc23:00:392
3347728993324,5cyclictest3408200-21systemd-run21:40:022
3347728993322,8cyclictest3385383-21timerwakeupswit21:10:432
334772899332,23cyclictest3417364-21ssh21:50:312
3347727993326,4cyclictest1051-21mta-sts-daemon20:20:321
3347727993325,5cyclictest3505459-21cat23:40:001
3347727993325,5cyclictest3373206-21munin-plugin-st20:35:011
3347727993324,6cyclictest1457228-21cupsd22:09:411
3347726993328,2cyclictest0-21swapper/022:50:560
3347726993324,6cyclictest3452508-21kworker/u8:6-xprtiod22:45:230
3347726993323,7cyclictest3417269-21missed_timers21:50:280
3347728993225,4cyclictest3383909-21ssh21:10:022
3347728993225,3cyclictest0-21swapper/200:18:442
3347728993223,6cyclictest3514145-21aten_r2power_vo23:50:222
3347728993223,6cyclictest0-21swapper/221:00:002
3347727993225,3cyclictest0-21swapper/121:35:291
3347727993224,5cyclictest3347748-21latency_hist19:10:021
3347727993224,4cyclictest0-21swapper/121:15:381
3347727993223,6cyclictest420-21avahi-daemon00:33:351
3347726993228,2cyclictest0-21swapper/023:05:010
3347726993227,3cyclictest0-21swapper/022:40:220
3347726993226,3cyclictest0-21swapper/021:00:390
3347726993226,3cyclictest0-21swapper/000:10:260
3347726993226,3cyclictest0-21swapper/000:10:260
3347726993222,8cyclictest428-21mta-sts-daemon22:20:450
3347726993222,8cyclictest428-21mta-sts-daemon22:20:450
3347728993124,4cyclictest3473317-21latency_hist23:00:032
3347728993124,3cyclictest0-21swapper/222:50:222
3347728993123,5cyclictest3362701-21sendmail20:00:012
3347728993122,7cyclictest3512286-21sshd23:47:332
3347728993119,9cyclictest3522797-21needreboot00:00:292
3347727993125,3cyclictest0-21swapper/120:40:011
3347727993124,4cyclictest1055-21mta-sts-daemon00:13:491
3347727993124,4cyclictest1055-21mta-sts-daemon00:13:491
3347727993124,3cyclictest0-21swapper/123:15:431
3347727993124,3cyclictest0-21swapper/122:45:371
3347727993120,8cyclictest3368718-21idleruntime-cro20:20:011
3347726993122,6cyclictest0-21swapper/023:10:380
3347726993121,8cyclictest261-21systemd-journal19:25:010
3347726993121,6cyclictest3369536-21needreboot20:20:300
3347728993023,3cyclictest0-21swapper/222:00:262
3347728993021,6cyclictest3378233-21fschecks_time20:50:242
3347728993015,10cyclictest3447692-21sshd22:28:052
3347728993015,10cyclictest3447692-21sshd22:28:052
334772799303,12cyclictest3383707-21diskmemload00:20:591
3347727993023,4cyclictest3379113-21head20:50:421
3347727993023,4cyclictest1056-21mta-sts-daemon19:20:011
3347727993022,5cyclictest3523157-21mailstats00:00:391
3347727993022,5cyclictest1055-21mta-sts-daemon19:23:141
3347726993024,4cyclictest3554171-21ls00:40:010
3347726993023,5cyclictest3477559-21/usr/sbin/munin23:05:440
3347726993020,7cyclictest0-21swapper/022:15:010
3347726993018,9cyclictest0-21swapper/020:55:430
334772899298,4cyclictest3383707-21diskmemload21:25:352
3347728992922,4cyclictest3364549-21aten_r2power_fr20:05:222
3347728992922,3cyclictest0-21swapper/222:30:352
3347728992922,3cyclictest0-21swapper/222:30:352
3347728992918,8cyclictest3440669-21latency_hist22:20:022
3347727992923,3cyclictest0-21swapper/100:25:371
3347727992921,5cyclictest0-21swapper/119:15:011
3347727992916,7cyclictest3383707-21diskmemload23:10:311
3347726992923,4cyclictest3428931-21ssh22:05:230
3347726992923,3cyclictest1056-21mta-sts-daemon00:29:100
3347726992922,4cyclictest3513566-21cron23:50:010
3347727992820,5cyclictest3481221-21cron23:10:011
3347727992820,5cyclictest3364018-21timerwakeupswit20:00:421
3347726992822,4cyclictest3390872-21sh21:18:070
3347726992822,4cyclictest1054-21mta-sts-daemon23:24:200
3347726992821,5cyclictest3494348-21missed_timers23:25:290
3347728992719,5cyclictest3364222-21munin-plugin-st20:05:012
3347728992719,5cyclictest3350308-21systemctl19:15:392
3347727992722,4cyclictest3347725-21cyclictest20:45:311
3347727992721,3cyclictest0-21swapper/119:30:011
3347726992720,5cyclictest420-21avahi-daemon19:15:290
3347726992718,6cyclictest151ktimers/020:35:360
3347728992621,3cyclictest0-21swapper/220:20:012
3347728992620,3cyclictest0-21swapper/219:50:012
3347728992618,5cyclictest3372478-21missed_timers20:30:272
3347728992617,6cyclictest3349209-21munin-plugin-st19:15:012
3347727992620,4cyclictest3347725-21cyclictest20:10:211
3347727992618,5cyclictest3364582-21aten_r2power_po20:05:221
3347726992619,5cyclictest151ktimers/019:45:340
3347726992619,4cyclictest0-21swapper/021:35:300
3347726992616,8cyclictest3377592-21unixbench_multi20:45:430
334772899258,12cyclictest421-21dbus-daemon21:00:392
3347728992519,3cyclictest1055-21mta-sts-daemon20:20:422
3347728992517,5cyclictest3374702-21cut20:40:012
3347727992520,2cyclictest0-21swapper/121:05:001
3347727992517,5cyclictest3380635-21unixbench_singl20:55:431
3347726992521,2cyclictest0-21swapper/021:10:000
3347726992521,2cyclictest0-21swapper/019:25:330
3347726992520,3cyclictest3355220-21kworker/u8:1-xprtiod20:05:250
3347726992520,3cyclictest0-21swapper/020:00:430
3347726992520,2cyclictest0-21swapper/019:40:420
3347726992520,2cyclictest0-21swapper/019:40:420
3347726992517,6cyclictest3378975-21timerwakeupswit20:50:410
3347726992517,6cyclictest3366979-21timerwakeupswit20:10:430
3347726992516,7cyclictest3375558-21ntp_kernel_err20:40:290
3347726992516,7cyclictest3368612-21unixbench_singl20:15:420
3347726992512,10cyclictest3361194-21cron19:55:010
3347728992417,4cyclictest1051-21mta-sts-daemon20:25:322
334772899241,19cyclictest753-21ntpd19:25:352
3347727992416,4cyclictest28-21ksoftirqd/120:25:351
334772699242,19cyclictest1457231-21cups-browsed20:25:350
3347726992415,7cyclictest3354983-21timerwakeupswit19:30:420
3347728992315,5cyclictest3352222-21cron19:25:012
3347727992317,3cyclictest0-21swapper/120:45:011
334772899222,14cyclictest3366659-21ntp_states20:10:372
3347727992215,4cyclictest0-21swapper/119:50:021
334772699223,16cyclictest659-21snmpd19:55:350
3347728992112,6cyclictest421-21dbus-daemon19:10:012
3347726992115,4cyclictest0-21swapper/019:10:260
334772699191,9cyclictest1-21systemd19:10:010
334772999182,8cyclictest0-21swapper/321:55:383
334772999181,8cyclictest0-21swapper/300:20:333
334772999171,7cyclictest0-21swapper/321:20:343
334772999163,7cyclictest0-21swapper/323:35:343
334772999162,7cyclictest0-21swapper/323:15:373
334772999162,6cyclictest0-21swapper/321:15:393
334772999161,8cyclictest0-21swapper/300:25:323
334772999161,7cyclictest0-21swapper/322:40:293
334772999152,7cyclictest0-21swapper/300:30:373
334772999152,7cyclictest0-21swapper/300:05:343
334772999151,8cyclictest0-21swapper/300:15:333
334772999151,7cyclictest0-21swapper/323:05:393
334772999151,7cyclictest0-21swapper/322:25:313
334772999151,7cyclictest0-21swapper/322:25:313
334772999151,7cyclictest0-21swapper/300:35:313
334772999151,7cyclictest0-21swapper/300:00:303
334772999143,6cyclictest0-21swapper/322:05:323
334772999142,8cyclictest0-21swapper/322:55:373
334772999142,7cyclictest0-21swapper/323:30:343
334772999142,5cyclictest0-21swapper/320:25:303
334772999142,5cyclictest0-21swapper/320:10:363
334772999141,8cyclictest0-21swapper/322:00:313
334772999141,7cyclictest0-21swapper/322:50:333
334772999141,7cyclictest0-21swapper/322:20:323
334772999141,7cyclictest0-21swapper/322:20:323
334772999141,7cyclictest0-21swapper/322:10:343
334772999141,7cyclictest0-21swapper/321:45:313
334772999141,7cyclictest0-21swapper/321:35:333
334772999141,6cyclictest0-21swapper/320:40:303
334772999141,6cyclictest0-21swapper/320:00:303
334772999141,5cyclictest0-21swapper/320:35:293
334772999141,5cyclictest0-21swapper/319:30:293
334772999132,6cyclictest0-21swapper/320:35:013
334772999131,7cyclictest0-21swapper/323:50:363
334772999131,7cyclictest0-21swapper/321:30:263
334772999131,7cyclictest0-21swapper/321:15:023
334772999131,6cyclictest0-21swapper/323:20:323
334772999131,6cyclictest0-21swapper/320:45:363
334772999131,5cyclictest0-21swapper/319:50:323
334772999122,5cyclictest0-21swapper/319:45:003
334772999122,5cyclictest0-21swapper/319:45:003
334772999122,5cyclictest0-21swapper/300:10:383
334772999122,5cyclictest0-21swapper/300:10:383
334772999121,6cyclictest0-21swapper/320:50:303
334772999121,6cyclictest0-21swapper/320:20:393
334772999121,6cyclictest0-21swapper/319:30:013
334772999121,5cyclictest0-21swapper/321:05:323
334772999121,5cyclictest0-21swapper/320:05:343
334772999121,5cyclictest0-21swapper/319:45:303
334772999121,5cyclictest0-21swapper/319:15:293
334772999113,6cyclictest0-21swapper/323:00:233
334772999112,6cyclictest0-21swapper/323:55:313
334772999112,5cyclictest0-21swapper/321:25:213
334772999112,5cyclictest0-21swapper/319:55:243
334772999111,7cyclictest0-21swapper/323:10:213
334772999111,6cyclictest0-21swapper/323:45:203
334772999111,6cyclictest0-21swapper/323:25:263
334772999111,6cyclictest0-21swapper/322:30:233
334772999111,6cyclictest0-21swapper/322:30:233
334772999111,6cyclictest0-21swapper/321:40:223
334772999111,6cyclictest0-21swapper/321:00:393
334772999101,6cyclictest0-21swapper/323:40:323
334772999101,6cyclictest0-21swapper/322:45:243
334772999101,6cyclictest0-21swapper/322:35:213
334772999101,6cyclictest0-21swapper/321:50:363
334772999101,5cyclictest0-21swapper/321:00:003
334772999101,5cyclictest0-21swapper/320:15:353
334772999101,4cyclictest0-21swapper/319:35:253
33477299991,5cyclictest0-21swapper/319:10:243
33477299991,3cyclictest0-21swapper/319:20:253
33477299982,4cyclictest0-21swapper/322:15:213
33477299971,3cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional