You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 18:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue May 12, 2026 12:43:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3092789995545,6cyclictest3281895-21needreboot12:20:472
3092789995444,6cyclictest420-21avahi-daemon11:25:022
3092789995142,5cyclictest3184997-21ssh10:20:022
3092789995142,5cyclictest3184997-21ssh10:20:022
3092788995144,4cyclictest3201120-21latency_hist10:40:021
3092789994940,6cyclictest3169515-21aten_r2power_fr10:00:422
3092789994924,22cyclictest3250045-21if_wlan011:40:462
3092788994940,6cyclictest3268913-21latency_hist12:05:021
3092788994940,6cyclictest3136859-21latency_hist09:20:031
3092788994939,6cyclictest3280834-21latency_hist12:20:021
3092788994939,6cyclictest3152811-21sed09:40:031
3092788994939,6cyclictest3152811-21sed09:40:031
3092788994834,9cyclictest261-21systemd-journal09:54:061
3092789994733,8cyclictest1-21systemd09:20:022
3092789994726,18cyclictest0-21swapper/212:05:012
3092789994723,6cyclictest1051-21mta-sts-daemon11:16:042
3092789994722,3cyclictest0-21swapper/211:55:012
309278899475,34cyclictest3206483-21ssh10:45:541
3092787994740,5cyclictest3182829-21unixbench_multi10:16:040
3092787994740,5cyclictest3182829-21unixbench_multi10:16:040
309278999465,20cyclictest3174191-21ssh10:05:532
3092789994638,5cyclictest3280692-21cut12:20:012
3092787994638,5cyclictest1055-21mta-sts-daemon12:28:020
309278999453,36cyclictest3158157-21ssh09:45:562
3092789994519,7cyclictest3141249-21/usr/sbin/munin09:25:472
3092789994511,9cyclictest3239105-21/usr/sbin/munin11:26:082
3092789994511,9cyclictest3239105-21/usr/sbin/munin11:26:082
3092787994537,6cyclictest548-21gdbus11:00:560
3092789994411,11cyclictest421-21dbus-daemon11:45:472
3092789994335,5cyclictest421-21dbus-daemon11:35:012
3092789994335,5cyclictest421-21dbus-daemon11:35:012
3092789994335,4cyclictest0-21swapper/210:35:512
3092789994334,4cyclictest0-21swapper/210:25:522
3092789994332,7cyclictest420-21avahi-daemon11:37:542
3092789994323,17cyclictest0-21swapper/212:30:572
3092788994335,5cyclictest3103296-21systemd-run07:45:011
3092787994335,5cyclictest2254995-21cups-browsed10:34:540
3092787994321,7cyclictest12350irq/32-eth011:26:030
3092787994321,7cyclictest12350irq/32-eth011:26:030
309278999423,16cyclictest3202541-21ssh10:40:582
3092789994214,8cyclictest3226216-21if_wlan011:10:462
3092789994212,8cyclictest62450lldpd10:34:002
3092788994236,3cyclictest0-21swapper/110:00:001
3092787994228,7cyclictest3234622-21sh11:21:000
3092789994132,6cyclictest1-21systemd07:50:022
3092789994131,6cyclictest2254990-21cupsd10:21:412
3092789994117,5cyclictest0-21swapper/212:05:472
3092789994117,5cyclictest0-21swapper/212:05:472
3092788994133,5cyclictest3233260-21munin-run11:20:011
3092788994133,5cyclictest1-21systemd11:10:011
3092788994133,4cyclictest0-21swapper/112:30:011
3092787994135,4cyclictest1-21systemd12:00:010
3092789994034,3cyclictest0-21swapper/211:55:452
3092789994031,6cyclictest421-21dbus-daemon10:00:012
3092789994031,5cyclictest3119750-21cut08:40:002
3092788994033,4cyclictest0-21swapper/110:15:501
3092788994033,4cyclictest0-21swapper/110:15:501
3092788994030,7cyclictest3140699-21munin-plugin-st09:25:001
3092788994029,8cyclictest548-21gdbus11:20:591
3092788994027,10cyclictest3129612-21sh09:10:461
3092787994031,7cyclictest3284672-21cron12:25:010
3092787994029,9cyclictest0-21swapper/011:46:070
3092789993932,3cyclictest0-21swapper/211:05:012
3092789993930,6cyclictest420-21avahi-daemon12:40:012
3092789993929,6cyclictest2254990-21cupsd10:51:012
3092789993929,6cyclictest2254990-21cupsd10:51:012
3092789993927,9cyclictest3095081-21needreboot07:15:482
3092789993926,9cyclictest1-21systemd10:45:412
3092789993915,6cyclictest171rcu_preempt09:50:552
309278899393,30cyclictest3158017-21ssh09:45:501
3092787993933,3cyclictest0-21swapper/009:20:030
309278899383,30cyclictest3293875-21ssh12:35:531
3092788993829,6cyclictest3128743-21cron09:10:011
3092787993832,4cyclictest3198182-21missed_timers10:35:490
3092787993830,5cyclictest2254995-21cups-browsed11:05:560
3092789993730,4cyclictest3094263-21sh07:15:012
3092789993725,5cyclictest0-21swapper/209:00:532
3092789993717,3cyclictest0-21swapper/209:21:152
3092788993731,3cyclictest0-21swapper/112:35:011
3092788993729,4cyclictest0-21swapper/111:40:531
309278899372,27cyclictest3258245-21ssh11:50:521
3092787993729,6cyclictest3193027-21sh10:30:010
3092787993729,5cyclictest421-21dbus-daemon10:10:530
3092787993729,5cyclictest421-21dbus-daemon09:30:010
3092787993729,5cyclictest2254995-21cups-browsed12:00:170
3092787993715,18cyclictest3154079-21ssh09:40:530
3092789993629,3cyclictest0-21swapper/211:00:022
3092789993628,5cyclictest3113765-21systemctl08:20:012
3092789993626,6cyclictest3288670-21systemd-run12:30:012
3092789993626,6cyclictest3097300-21systemd-run07:25:012
3092788993629,4cyclictest3100271-21munin-plugin-st07:35:001
3092788993629,4cyclictest3100271-21munin-plugin-st07:35:001
3092788993629,4cyclictest0-21swapper/108:50:011
3092788993628,5cyclictest3256938-21munin-plugin-st11:50:011
3092788993628,5cyclictest3210710-21mailstats10:51:001
3092788993628,5cyclictest3210710-21mailstats10:51:001
3092788993628,5cyclictest3097276-21cron07:25:011
3092788993628,5cyclictest1-21systemd12:25:021
3092788993627,6cyclictest420-21avahi-daemon12:06:001
3092788993627,6cyclictest420-21avahi-daemon12:06:001
3092787993627,6cyclictest2254990-21cupsd10:07:230
3092787993613,14cyclictest3210484-21ssh10:50:540
3092787993613,14cyclictest3210484-21ssh10:50:540
3092789993527,5cyclictest3109277-21idleruntime-cro08:05:012
3092789993526,6cyclictest3153274-21/usr/sbin/munin09:40:472
309278999351,25cyclictest3274160-21ssh12:10:542
3092788993529,3cyclictest0-21swapper/112:10:531
3092788993526,6cyclictest2254995-21cups-browsed09:31:361
309278799353,25cyclictest3258283-21ssh11:50:530
3092787993528,5cyclictest420-21avahi-daemon10:05:010
3092787993527,5cyclictest2254995-21cups-browsed09:23:070
3092787993526,6cyclictest420-21avahi-daemon09:59:100
3092789993427,3cyclictest0-21swapper/210:11:062
3092789993426,4cyclictest0-21swapper/209:30:512
309278799347,18cyclictest428-21mta-sts-daemon07:30:570
309278799347,18cyclictest428-21mta-sts-daemon07:30:570
309278799341,28cyclictest3162198-21ssh09:50:570
309278999334,23cyclictest2254995-21cups-browsed09:36:012
309278999334,23cyclictest2254995-21cups-browsed09:36:012
3092789993326,3cyclictest0-21swapper/208:30:012
3092789993325,5cyclictest3223076-21unixbench-2d11:06:042
3092788993329,2cyclictest0-21swapper/108:00:011
309278899332,23cyclictest3154088-21ssh09:40:541
3092788993318,11cyclictest421-21dbus-daemon11:35:531
3092787993327,4cyclictest3202957-21unixbench-2d10:41:040
3092787993325,5cyclictest3274711-21unixbench_singl12:11:030
309278799332,25cyclictest151ktimers/008:50:520
3092788993227,2cyclictest0-21swapper/110:10:421
3092788993226,3cyclictest0-21swapper/110:56:041
3092788993226,3cyclictest0-21swapper/110:30:401
3092788993225,4cyclictest1055-21mta-sts-daemon08:20:561
3092788993223,6cyclictest421-21dbus-daemon10:40:461
3092788993222,8cyclictest3170554-21switchtime10:01:041
3092788993222,7cyclictest421-21dbus-daemon11:25:461
3092788993222,7cyclictest421-21dbus-daemon11:25:461
3092788993222,7cyclictest3116871-21latency_hist08:30:021
309278899322,23cyclictest3262165-21ssh11:55:521
3092787993227,3cyclictest3092786-21cyclictest10:55:570
3092787993225,4cyclictest3092798-21systemd-run07:10:010
3092787993225,3cyclictest0-21swapper/009:10:010
3092787993224,5cyclictest2254995-21cups-browsed09:12:170
3092787993223,7cyclictest3245067-21cron11:35:000
3092787993223,7cyclictest3245067-21cron11:35:000
3092787993221,3cyclictest0-21swapper/011:19:530
309278799321,11cyclictest171rcu_preempt10:20:510
3092789993122,5cyclictest3103346-21latency_hist07:45:012
3092789993121,7cyclictest3118249-21cron08:35:002
3092789993121,7cyclictest3118249-21cron08:35:002
3092789993120,7cyclictest3129523-21aten2.4-expect09:10:442
3092788993124,4cyclictest3093419-21kernelversion07:10:471
3092788993124,3cyclictest0-21swapper/110:21:011
3092788993123,5cyclictest3112277-21systemd-run08:15:011
309278899312,22cyclictest3226573-21ssh11:10:531
3092787993125,3cyclictest3122748-21cron08:50:010
3092787993123,5cyclictest2254995-21cups-browsed09:31:580
3092787993122,6cyclictest0-21swapper/011:45:000
3092787993121,4cyclictest3270477-21switchtime12:06:040
3092787993121,4cyclictest3270477-21switchtime12:06:040
3092787993120,8cyclictest1-21systemd07:25:020
3092787993120,6cyclictest0-21swapper/010:45:510
3092787993115,12cyclictest151ktimers/009:36:000
3092787993115,12cyclictest151ktimers/009:36:000
309278999302,23cyclictest659-21snmpd08:50:522
3092789993019,8cyclictest3122073-21needreboot08:45:482
3092788993023,5cyclictest0-21swapper/111:05:011
3092788993023,4cyclictest0-21swapper/108:20:011
3092788993023,3cyclictest0-21swapper/111:30:521
3092788993023,3cyclictest0-21swapper/111:30:521
3092788993023,3cyclictest0-21swapper/109:25:511
3092788993022,5cyclictest3119325-21sendmail_mailst08:35:561
3092788993021,6cyclictest3189873-21fschecks_count10:25:441
3092787993024,3cyclictest0-21swapper/009:00:010
3092787993023,4cyclictest439-21systemd-logind08:45:010
3092787993023,4cyclictest3183456-21kworker/u8:0-writeback11:10:470
3092787993021,6cyclictest3158475-21timerandwakeup09:46:020
3092787993021,6cyclictest3114046-21aten_r2power_cu08:20:390
3092787993021,6cyclictest0-21swapper/008:15:010
3092787993021,5cyclictest1051-21mta-sts-daemon08:00:540
309278899299,14cyclictest420-21avahi-daemon07:15:561
3092788992921,5cyclictest3101769-21systemctl07:40:001
3092788992921,5cyclictest2254995-21cups-browsed07:25:171
3092788992917,9cyclictest3174313-21ntp_states10:05:581
3092787992921,5cyclictest261-21systemd-journal07:40:000
3092787992920,6cyclictest3095737-21kworker/u8:7-xprtiod08:10:010
3092787992918,7cyclictest3245705-21aten2.4-expect11:35:420
3092789992821,4cyclictest3078304-21kworker/u8:1-xprtiod08:00:012
3092789992821,3cyclictest0-21swapper/209:00:012
3092789992820,5cyclictest0-21swapper/207:35:012
3092789992820,5cyclictest0-21swapper/207:35:012
3092788992823,2cyclictest0-21swapper/108:10:011
3092788992821,4cyclictest3109168-21grep08:01:061
3092787992823,3cyclictest0-21swapper/007:55:020
3092787992818,6cyclictest3264887-21kworker/u8:1-xprtiod12:15:400
3092787992818,5cyclictest3107684-21unixbench_singl07:56:010
3092788992722,2cyclictest0-21swapper/109:00:011
3092788992720,5cyclictest3127069-21timerwakeupswit09:01:051
3092788992710,3cyclictest271ktimers/107:45:511
309278799277,17cyclictest3293872-21sh12:35:530
309278799275,14cyclictest428-21mta-sts-daemon07:25:570
3092787992721,3cyclictest695-21cron08:40:000
3092788992618,5cyclictest3124241-21latency_hist08:55:011
3092787992620,4cyclictest3116913-21latency_hist08:30:020
3092787992620,4cyclictest2254995-21cups-browsed08:31:010
3092787992620,4cyclictest2254995-21cups-browsed08:31:010
3092787992612,10cyclictest12350irq/32-eth007:15:560
3092789992519,3cyclictest0-21swapper/209:10:012
3092789992517,5cyclictest3110766-21munin-plugin-st08:10:012
3092787992516,7cyclictest3289955-21ntp_kernel_pll_12:30:520
3092789992418,3cyclictest0-21swapper/207:35:522
3092789992417,4cyclictest3114576-21grep08:20:492
3092789992416,5cyclictest439-21systemd-logind07:10:022
3092788992416,5cyclictest3118106-21unixbench-2d08:31:001
3092788992416,5cyclictest3118106-21unixbench-2d08:31:001
3092787992417,5cyclictest439-21systemd-logind08:20:010
3092787992417,5cyclictest3126128-21telnet09:00:430
3092787992414,4cyclictest3102308-21fschecks_time07:40:480
3092789992316,5cyclictest3120825-21sendmail_mailqu08:40:572
3092789992313,7cyclictest421-21dbus-daemon07:25:052
3092788992313,8cyclictest3105608-21ntp_kernel_err07:50:511
309278799233,5cyclictest3104258-21ntpq07:45:580
3092787992316,5cyclictest3093110-21aten_r2power_fr07:10:420
3092789992211,8cyclictest547-21gmain07:51:032
309278899224,13cyclictest421-21dbus-daemon08:40:551
3092789992115,3cyclictest0-21swapper/208:10:512
309279099191,13cyclictest0-21swapper/309:40:493
3092788991914,2cyclictest0-21swapper/107:10:021
309279099181,11cyclictest0-21swapper/312:25:493
309279099172,10cyclictest0-21swapper/312:10:533
309279099171,12cyclictest0-21swapper/311:45:523
309279099171,12cyclictest0-21swapper/310:45:563
309279099171,12cyclictest0-21swapper/309:50:503
309279099171,11cyclictest0-21swapper/312:30:493
309279099171,11cyclictest0-21swapper/309:55:503
309279099171,11cyclictest0-21swapper/309:45:513
309279099170,12cyclictest0-21swapper/309:20:593
309279099162,10cyclictest0-21swapper/309:05:533
309279099161,13cyclictest0-21swapper/307:10:013
309279099161,11cyclictest0-21swapper/312:20:513
309279099161,11cyclictest0-21swapper/311:52:153
309279099161,11cyclictest0-21swapper/311:35:523
309279099161,11cyclictest0-21swapper/311:20:543
309279099161,11cyclictest0-21swapper/311:15:553
309279099161,11cyclictest0-21swapper/311:05:523
309279099161,11cyclictest0-21swapper/311:00:513
309279099161,11cyclictest0-21swapper/310:30:513
309279099161,11cyclictest0-21swapper/310:05:533
309279099161,11cyclictest0-21swapper/310:00:523
309279099161,11cyclictest0-21swapper/309:30:513
309279099161,11cyclictest0-21swapper/309:25:523
309279099161,11cyclictest0-21swapper/308:55:503
309279099161,11cyclictest0-21swapper/308:35:553
309279099161,11cyclictest0-21swapper/308:10:493
309279099161,11cyclictest0-21swapper/308:05:523
309279099161,11cyclictest0-21swapper/308:00:543
309279099161,11cyclictest0-21swapper/307:55:493
309279099161,11cyclictest0-21swapper/307:35:513
309279099161,11cyclictest0-21swapper/307:30:543
309279099161,11cyclictest0-21swapper/307:30:543
309279099161,11cyclictest0-21swapper/307:25:573
309279099160,11cyclictest0-21swapper/309:00:533
309279099151,11cyclictest0-21swapper/312:05:553
309279099151,11cyclictest0-21swapper/312:05:553
309279099151,11cyclictest0-21swapper/312:00:013
309279099151,11cyclictest0-21swapper/311:40:413
309279099151,11cyclictest0-21swapper/310:55:433
309279099151,11cyclictest0-21swapper/310:50:533
309279099151,11cyclictest0-21swapper/310:50:533
309279099151,11cyclictest0-21swapper/310:40:563
309279099151,11cyclictest0-21swapper/310:20:583
309279099151,11cyclictest0-21swapper/310:15:413
309279099151,11cyclictest0-21swapper/310:15:413
309279099151,11cyclictest0-21swapper/310:10:513
309279099151,11cyclictest0-21swapper/309:15:503
309279099151,10cyclictest0-21swapper/312:35:173
309279099151,10cyclictest0-21swapper/311:30:413
309279099151,10cyclictest0-21swapper/311:30:413
309279099151,10cyclictest0-21swapper/311:25:513
309279099151,10cyclictest0-21swapper/311:25:513
309279099151,10cyclictest0-21swapper/310:35:393
309279099151,10cyclictest0-21swapper/308:50:513
309279099151,10cyclictest0-21swapper/308:45:493
309279099151,10cyclictest0-21swapper/307:45:533
309279099151,10cyclictest0-21swapper/307:40:543
309279099151,10cyclictest0-21swapper/307:10:513
309279099150,11cyclictest0-21swapper/312:15:323
309279099150,11cyclictest0-21swapper/309:35:573
309279099150,11cyclictest0-21swapper/309:35:573
309279099150,11cyclictest0-21swapper/309:10:533
309279099150,11cyclictest0-21swapper/308:40:553
309279099150,11cyclictest0-21swapper/308:15:513
309279099141,10cyclictest0-21swapper/311:10:403
309279099141,10cyclictest0-21swapper/310:25:413
309279099141,10cyclictest0-21swapper/308:30:443
309279099141,10cyclictest0-21swapper/308:30:443
309279099141,10cyclictest0-21swapper/308:25:483
309279099141,10cyclictest0-21swapper/308:20:463
309279099141,10cyclictest0-21swapper/307:50:423
309279099141,10cyclictest0-21swapper/307:20:423
309279099141,10cyclictest0-21swapper/307:15:493
309279099140,10cyclictest0-21swapper/312:00:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional