You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 13:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Wed Apr 01, 2026 00:43:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3311432995544,7cyclictest3487818-21cron00:00:001
331143299549,40cyclictest3390119-21ssh22:00:341
3311432995345,3cyclictest0-21swapper/122:40:221
3311432995238,9cyclictest1054-21mta-sts-daemon23:50:271
3311433995145,3cyclictest0-21swapper/222:15:312
3311432995143,4cyclictest0-21swapper/123:10:271
3311433995041,6cyclictest420-21avahi-daemon22:20:372
3311431995037,5cyclictest12350irq/32-eth021:50:300
3311431995037,5cyclictest12350irq/32-eth021:50:300
3311431995018,29cyclictest659-21snmpd23:20:330
3311433994840,5cyclictest3498906-21idleruntime-cro00:15:012
3311431994840,6cyclictest439-21systemd-logind00:10:020
3311432994739,5cyclictest3405478-21awk22:20:011
331143299464,33cyclictest3435745-21ssh22:55:341
3311432994638,5cyclictest3507136-21cron00:25:011
331143199462,15cyclictest151ktimers/020:40:280
3311433994536,6cyclictest3471409-21sendmail23:40:012
3311433994535,6cyclictest1360225-21cups-browsed23:42:042
3311433994535,6cyclictest1360225-21cups-browsed23:42:042
3311432994535,6cyclictest3496106-21proc_pri00:10:331
331143299452,34cyclictest3452076-21ssh23:15:291
331143199453,39cyclictest0-21swapper/022:20:340
3311433994435,6cyclictest3459160-21true23:25:012
3311433994423,14cyclictest351ktimers/219:55:342
3311432994434,7cyclictest3372217-21systemctl21:40:011
3311432994334,6cyclictest516-21in:imuxsock19:15:021
3311433994235,5cyclictest3366314-21sshd21:31:482
3311433994233,5cyclictest3481330-21unixbench_multi23:50:392
3311433994231,7cyclictest421-21dbus-daemon22:25:362
3311433994230,8cyclictest420-21avahi-daemon19:40:282
3311433994226,11cyclictest351ktimers/223:05:302
331143299423,23cyclictest1051-21mta-sts-daemon21:45:331
3311433994131,7cyclictest420-21avahi-daemon23:56:112
3311432994132,5cyclictest3516460-21ntp_kernel_pll_00:35:291
3311431994134,5cyclictest3338486-21systemd-run20:40:010
3311433994034,3cyclictest0-21swapper/222:40:012
3311433994032,5cyclictest3345991-21systemd-run21:05:022
3311433994032,3cyclictest0-21swapper/223:00:292
331143399402,32cyclictest3512363-21ssh00:30:292
331143399402,32cyclictest3512363-21ssh00:30:292
3311432994032,4cyclictest0-21swapper/122:05:311
3311432994031,4cyclictest0-21swapper/121:40:271
331143299401,33cyclictest3427299-21ssh22:45:271
3311431994034,4cyclictest421-21dbus-daemon22:50:010
3311431994032,5cyclictest3442757-21systemd-run23:05:010
3311433993934,2cyclictest0-21swapper/220:20:012
3311433993931,3cyclictest0-21swapper/221:55:292
3311433993931,3cyclictest0-21swapper/221:55:292
3311433993930,6cyclictest3393035-21idleruntime-cro22:05:002
3311433993929,6cyclictest3467279-21cron23:35:012
3311433993922,13cyclictest261-21systemd-journal23:15:282
3311432993932,3cyclictest0-21swapper/123:35:341
3311432993930,4cyclictest0-21swapper/121:55:321
3311432993930,4cyclictest0-21swapper/121:55:321
3311432993929,7cyclictest3472193-21/usr/sbin/munin23:40:251
3311432993929,7cyclictest3472193-21/usr/sbin/munin23:40:251
3311432993929,6cyclictest3357473-21unixbench_singl21:20:411
331143199393,27cyclictest3410694-21ssh22:25:280
3311431993932,5cyclictest3432050-21grep22:50:410
3311433993832,3cyclictest0-21swapper/200:40:002
3311433993831,3cyclictest0-21swapper/200:10:012
3311433993829,4cyclictest0-21swapper/219:25:322
331143299383,29cyclictest3431611-21ssh22:50:331
3311432993830,5cyclictest3488262-21aten_r2power_cu00:00:181
3311432993829,4cyclictest0-21swapper/119:35:311
331143199381,28cyclictest3443924-21ssh23:05:320
3311433993730,4cyclictest0-21swapper/200:20:362
3311433993730,3cyclictest0-21swapper/220:15:022
3311433993729,4cyclictest0-21swapper/222:15:012
3311433993728,5cyclictest421-21dbus-daemon22:10:022
3311433993728,5cyclictest3326486-21kworker/u8:13+xprtiod21:10:012
3311433993728,5cyclictest0-21swapper/222:30:332
3311433993728,4cyclictest0-21swapper/221:10:362
3311433993728,4cyclictest0-21swapper/221:10:362
3311432993731,3cyclictest3440595-21kworker/u8:1-rpciod23:25:261
3311431993731,4cyclictest1-21systemd19:20:020
3311431993730,4cyclictest3515361-21systemd-run00:35:020
3311431993730,4cyclictest3515361-21systemd-run00:35:020
3311431993729,6cyclictest3321985-21systemd-run19:45:010
3311431993727,7cyclictest12350irq/32-eth023:45:300
3311431993727,7cyclictest12350irq/32-eth023:45:300
3311431993716,15cyclictest12350irq/32-eth000:15:360
3311433993627,6cyclictest3372665-21aten_r2power_en21:40:172
3311433993626,4cyclictest0-21swapper/222:55:352
3311433993625,8cyclictest3377412-21needreboot21:45:272
3311432993628,5cyclictest695-21cron00:30:011
3311432993627,6cyclictest3322496-21fschecks_time19:45:211
3311431993629,4cyclictest421-21dbus-daemon22:15:190
3311431993625,9cyclictest3415433-21ssh22:30:410
3311431993616,14cyclictest421-21dbus-daemon19:20:290
331143399354,26cyclictest3356787-21ssh21:20:282
331143399353,26cyclictest3447963-21ssh23:10:292
3311433993528,3cyclictest0-21swapper/221:15:282
3311433993527,4cyclictest0-21swapper/200:15:222
331143399352,6cyclictest171rcu_preempt00:25:292
331143399352,23cyclictest3360857-21sh21:25:272
3311433993513,4cyclictest0-21swapper/220:05:342
3311432993528,3cyclictest0-21swapper/122:35:281
3311432993524,4cyclictest1054-21mta-sts-daemon21:12:301
3311432993524,4cyclictest1054-21mta-sts-daemon21:12:301
3311432993522,7cyclictest3406639-21sh22:20:301
331143199354,27cyclictest3496056-21ssh00:10:320
3311431993528,3cyclictest0-21swapper/021:45:350
3311431993527,5cyclictest12350irq/32-eth022:35:280
3311431993526,6cyclictest3484595-21latency23:55:250
3311431993523,9cyclictest3401315-21idleruntime-cro22:15:010
3311431993521,11cyclictest0-21swapper/019:40:010
3311431993516,11cyclictest12350irq/32-eth000:00:300
331143199351,29cyclictest3390110-21ssh22:00:330
3311433993427,3cyclictest0-21swapper/223:45:282
3311433993427,3cyclictest0-21swapper/223:45:282
3311433993426,5cyclictest3324971-21systemd-run19:55:012
3311433993426,5cyclictest3324971-21systemd-run19:55:012
3311433993426,3cyclictest0-21swapper/223:30:022
3311432993428,3cyclictest0-21swapper/120:15:011
3311432993427,4cyclictest1055-21mta-sts-daemon21:33:561
3311432993426,4cyclictest0-21swapper/122:25:291
3311432993414,15cyclictest3381792-21ntpq21:50:351
3311432993414,15cyclictest3381792-21ntpq21:50:351
3311431993428,4cyclictest3455017-21cron23:20:010
331143399334,17cyclictest1055-21mta-sts-daemon20:40:272
3311433993326,4cyclictest1055-21mta-sts-daemon21:35:032
3311433993325,5cyclictest3342946-21idleruntime-cro20:55:002
3311433993325,5cyclictest3318981-21systemd-run19:35:022
3311433993323,6cyclictest659-21snmpd21:50:292
3311433993323,6cyclictest659-21snmpd21:50:292
3311433993321,8cyclictest421-21dbus-daemon20:50:012
3311433993321,8cyclictest421-21dbus-daemon20:50:012
3311433993319,10cyclictest3347461-21diskmemload22:40:162
331143299333,25cyclictest3439954-21ssh23:00:371
3311432993327,2cyclictest0-21swapper/122:15:011
3311432993326,4cyclictest1055-21mta-sts-daemon23:46:361
3311432993326,4cyclictest1055-21mta-sts-daemon23:46:361
3311432993326,3cyclictest0-21swapper/100:15:341
3311432993325,5cyclictest3336377-21ntp_kernel_pll_20:30:301
331143299332,28cyclictest3443897-21ssh23:05:311
331143299331,24cyclictest3512487-21sh00:30:341
331143299331,24cyclictest3512487-21sh00:30:341
331143199334,22cyclictest3435775-21rm22:55:350
3311431993325,6cyclictest1-21systemd00:20:340
3311431993325,5cyclictest3356125-21tail21:20:180
3311431993324,7cyclictest3480449-21kernelversion23:50:230
3311431993323,7cyclictest3422925-21kernelversion22:40:230
3311431993318,10cyclictest3508258-21ssh00:25:300
331143399323,23cyclictest3431539-21ssh22:50:312
331143399322,22cyclictest3427347-21ssh22:45:292
331143399322,21cyclictest3343812-21sshd20:55:282
3311432993226,3cyclictest0-21swapper/121:10:011
3311432993222,7cyclictest3327970-21/usr/sbin/munin20:05:011
3311432993222,7cyclictest1054-21mta-sts-daemon00:09:431
3311432993222,6cyclictest3335462-21latency_hist20:30:011
331143299321,25cyclictest3464482-21ssh23:30:351
3311431993226,4cyclictest3394732-21sed22:05:430
3311431993225,4cyclictest3368137-21latency_hist21:35:020
3311433993125,3cyclictest0-21swapper/219:10:282
3311433993122,5cyclictest3487809-21cups-browsed00:00:032
3311433993120,8cyclictest3319855-21ntp_kernel_pll_19:35:302
3311433993119,9cyclictest3313797-21ntp_kernel_err19:15:272
3311432993125,3cyclictest0-21swapper/121:30:011
3311432993123,5cyclictest3342951-21idleruntime-cro20:55:001
3311432993123,5cyclictest3339965-21munin-plugin-st20:45:001
3311431993126,2cyclictest0-21swapper/021:15:270
3311431993125,4cyclictest3324302-21sshd19:50:270
3311431993125,4cyclictest3324302-21sshd19:50:270
3311431993125,3cyclictest1055-21mta-sts-daemon23:15:020
3311431993124,4cyclictest1-21systemd20:35:020
3311431993122,6cyclictest0-21swapper/023:25:250
3311431993121,7cyclictest3373945-21unixbench_multi21:40:390
3311431993121,4cyclictest3368746-21fschecks_count21:35:210
3311431993116,10cyclictest12350irq/32-eth023:40:350
3311431993116,10cyclictest12350irq/32-eth023:40:350
3311433993023,4cyclictest1051-21mta-sts-daemon20:00:322
3311433993020,8cyclictest3333317-21ntp_kernel_err20:20:272
331143299303,20cyclictest3415008-21ssh22:30:341
3311432993023,4cyclictest3313596-21kernelversion19:15:241
3311432993023,3cyclictest0-21swapper/121:00:301
3311432993022,5cyclictest3338449-21idleruntime-cro20:40:011
3311432993022,3cyclictest0-21swapper/123:20:331
3311431993024,4cyclictest1360225-21cups-browsed23:35:280
3311431993020,4cyclictest1056-21mta-sts-daemon20:45:180
3311431993020,4cyclictest1056-21mta-sts-daemon20:45:180
3311431993019,6cyclictest12350irq/32-eth023:30:280
3311432992922,4cyclictest1056-21mta-sts-daemon21:15:281
3311432992922,4cyclictest0-21swapper/120:00:011
3311432992921,5cyclictest421-21dbus-daemon19:50:331
3311432992921,5cyclictest421-21dbus-daemon19:50:331
3311432992921,5cyclictest3315972-21cron19:25:011
3311432992920,6cyclictest3340982-21proc_pri20:45:341
3311432992920,6cyclictest3340982-21proc_pri20:45:341
3311433992822,3cyclictest3315963-21kworker/u8:5-xprtiod19:50:002
3311432992820,5cyclictest3332875-21aten_r2power_vo20:20:201
3311432992819,6cyclictest3317975-21fschecks_count19:30:231
3311432992817,8cyclictest3343845-21ntp_kernel_pll_20:55:301
3311431992822,3cyclictest0-21swapper/021:55:410
3311431992822,3cyclictest0-21swapper/021:55:410
3311431992820,6cyclictest3332255-21timerwakeupswit20:15:390
3311431992820,6cyclictest3318854-21unixbench_multi19:30:410
3311431992723,2cyclictest0-21swapper/019:30:010
3311431992721,3cyclictest0-21swapper/000:35:310
3311431992719,5cyclictest3348109-21cpuspeed21:10:210
3311431992719,5cyclictest3348109-21cpuspeed21:10:210
3311433992619,3cyclictest0-21swapper/219:25:012
3311432992610,13cyclictest271ktimers/119:40:321
3311431992622,2cyclictest0-21swapper/021:25:360
3311431992620,4cyclictest3335398-21unixbench_singl20:25:420
3311431992618,5cyclictest3312984-21systemd-run19:15:020
3311433992520,2cyclictest0-21swapper/220:25:322
3311433992518,4cyclictest421-21dbus-daemon19:10:012
3311433992517,5cyclictest3338441-21cron20:40:012
3311431992518,5cyclictest1360225-21cups-browsed21:04:300
3311431992510,11cyclictest605-21lldpd20:05:310
3311433992416,4cyclictest0-21swapper/220:30:322
3311432992416,5cyclictest3317341-21unixbench_multi19:25:391
331143199242,17cyclictest3343814-21modprobe20:55:280
331143199242,16cyclictest3330339-21ntpq20:10:280
3311431992417,5cyclictest0-21swapper/019:10:010
3311432992317,3cyclictest0-21swapper/120:15:401
3311432992317,3cyclictest0-21swapper/120:05:331
3311431992318,3cyclictest1055-21mta-sts-daemon20:00:210
3311431992310,10cyclictest3323464-21munin-plugin-st19:50:000
3311432992217,2cyclictest0-21swapper/119:10:011
331143199221,6cyclictest171rcu_preempt19:55:340
3311431992216,4cyclictest3333005-21fschecks_time20:20:220
331143199221,17cyclictest3346872-21ntpq21:05:270
3311431992210,3cyclictest171rcu_preempt20:50:290
331143499213,8cyclictest0-21swapper/322:15:253
331143499213,10cyclictest0-21swapper/300:15:293
331143499212,10cyclictest0-21swapper/322:25:323
331143499204,9cyclictest0-21swapper/322:20:283
331143499203,8cyclictest0-21swapper/322:55:313
331143499203,10cyclictest0-21swapper/319:30:323
331143499202,10cyclictest0-21swapper/321:15:283
331143499202,10cyclictest0-21swapper/300:20:283
331143499194,7cyclictest0-21swapper/320:55:313
331143499193,9cyclictest0-21swapper/323:30:293
331143499193,9cyclictest0-21swapper/321:25:303
331143499192,11cyclictest0-21swapper/322:05:323
331143499192,10cyclictest0-21swapper/300:25:313
331143499191,9cyclictest0-21swapper/323:50:303
331143499191,10cyclictest0-21swapper/320:05:323
331143499184,8cyclictest0-21swapper/323:40:313
331143499184,8cyclictest0-21swapper/323:40:313
331143499184,7cyclictest0-21swapper/321:45:333
331143499184,10cyclictest0-21swapper/320:15:293
331143499183,9cyclictest0-21swapper/322:00:273
331143499183,9cyclictest0-21swapper/300:00:283
331143499183,8cyclictest0-21swapper/322:40:293
331143499183,10cyclictest0-21swapper/322:36:283
331143499182,8cyclictest0-21swapper/300:35:293
331143499182,10cyclictest0-21swapper/321:10:343
331143499182,10cyclictest0-21swapper/321:10:343
331143499181,9cyclictest0-21swapper/323:45:263
331143499181,9cyclictest0-21swapper/323:45:263
331143499181,8cyclictest0-21swapper/300:10:253
331143499181,10cyclictest0-21swapper/323:25:323
331143499181,10cyclictest0-21swapper/300:30:283
331143499181,10cyclictest0-21swapper/300:30:283
331143499175,7cyclictest0-21swapper/322:30:293
331143499173,9cyclictest0-21swapper/323:15:303
331143499173,9cyclictest0-21swapper/323:05:303
331143499173,9cyclictest0-21swapper/321:55:293
331143499173,9cyclictest0-21swapper/321:55:293
331143499173,8cyclictest0-21swapper/322:10:283
331143499173,7cyclictest0-21swapper/323:10:303
331143499173,7cyclictest0-21swapper/321:50:193
331143499173,7cyclictest0-21swapper/321:50:193
331143499172,9cyclictest0-21swapper/323:35:313
331143499172,9cyclictest0-21swapper/323:20:283
331143499172,9cyclictest0-21swapper/321:35:273
331143499172,10cyclictest0-21swapper/323:00:323
331143499172,10cyclictest0-21swapper/319:40:323
331143499171,8cyclictest0-21swapper/319:15:343
331143499165,7cyclictest0-21swapper/323:55:313
331143499163,8cyclictest0-21swapper/322:50:323
331143499163,7cyclictest0-21swapper/319:20:233
331143499162,9cyclictest0-21swapper/322:45:303
331143499162,9cyclictest0-21swapper/321:30:313
331143499162,9cyclictest0-21swapper/321:20:313
331143499162,7cyclictest0-21swapper/320:35:283
331143499161,9cyclictest0-21swapper/321:40:263
331143499161,6cyclictest0-21swapper/319:55:293
331143499153,7cyclictest0-21swapper/321:05:253
331143499153,6cyclictest0-21swapper/319:50:303
331143499153,6cyclictest0-21swapper/319:50:303
331143499152,7cyclictest0-21swapper/319:10:273
331143499151,8cyclictest0-21swapper/320:30:203
331143499151,7cyclictest0-21swapper/320:10:323
331143499151,7cyclictest0-21swapper/319:35:293
331143499144,5cyclictest0-21swapper/319:25:313
331143499142,8cyclictest0-21swapper/300:05:343
331143499141,8cyclictest0-21swapper/319:45:273
331143499132,7cyclictest0-21swapper/320:50:273
331143499132,7cyclictest0-21swapper/320:45:273
331143499132,7cyclictest0-21swapper/320:45:273
331143499131,7cyclictest0-21swapper/320:20:293
331143499131,6cyclictest0-21swapper/320:25:293
331143499121,8cyclictest0-21swapper/320:05:003
331143499121,7cyclictest0-21swapper/320:40:263
331143499121,6cyclictest0-21swapper/321:00:283
331143499101,5cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional