You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 08:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Sep 03, 2026 00:42:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3387708995744,4cyclictest171rcu_preempt22:46:122
3387706995639,6cyclictest3527485-21systemctl00:00:010
338770699562,49cyclictest3437220-21sshd21:31:060
3387706995040,5cyclictest3551920-21latency_hist00:40:030
3387706995040,5cyclictest3551920-21latency_hist00:40:030
3387708994939,6cyclictest3475716-21latency_hist22:35:022
3387708994840,5cyclictest3460401-21ls22:10:012
3387707994839,6cyclictest1656768-21cups-browsed23:16:121
338770899464,31cyclictest3498352-21ssh23:11:162
3387708994536,6cyclictest421-21dbus-daemon19:35:012
3387708994536,6cyclictest421-21dbus-daemon19:35:012
3387707994429,11cyclictest3535172-21switchtime00:11:181
338770699441,37cyclictest12650irq/32-eth000:16:060
3387708994334,5cyclictest3544561-21tail00:26:192
3387707994336,4cyclictest3401195-21munin-plugin-st19:55:011
338770899423,28cyclictest3510518-21ssh23:31:092
338770899422,32cyclictest265-21systemd-journal23:41:072
3387707994235,4cyclictest0-21swapper/100:01:101
3387706994232,5cyclictest0-21swapper/022:46:100
3387708994131,6cyclictest3489593-21grep22:56:192
3387708994131,5cyclictest0-21swapper/223:35:592
3387707994133,5cyclictest3550610-21grep00:36:191
3387707994133,5cyclictest3550610-21grep00:36:191
3387707994132,5cyclictest3435988-21systemd-run21:30:011
3387706994133,5cyclictest3494772-21fschecks_count23:05:580
3387706994133,5cyclictest3494772-21fschecks_count23:05:580
3387706994133,5cyclictest3487880-21systemd22:55:020
3387708994027,6cyclictest0-21swapper/221:01:092
3387706994032,5cyclictest665-21snmpd22:41:190
3387706994031,6cyclictest3506752-21aten_r2power_en23:25:560
3387708993931,5cyclictest3492726-21unixbench_singl23:01:182
338770899392,29cyclictest3476841-21ssh22:36:052
338770899391,31cyclictest3504430-21ssh23:21:112
3387707993930,4cyclictest0-21swapper/122:21:061
3387707993930,4cyclictest0-21swapper/122:11:141
3387706993929,7cyclictest3520096-21timerwakeupswit23:46:180
3387708993832,3cyclictest0-21swapper/200:20:012
3387708993830,5cyclictest3551971-21systemd-run00:40:022
3387708993830,5cyclictest3551971-21systemd-run00:40:022
3387708993830,5cyclictest3534508-21ssh00:11:012
3387708993828,6cyclictest3437099-21needreboot21:31:022
3387708993819,3cyclictest171rcu_preempt21:21:142
3387708993817,6cyclictest3486077-21ssh22:51:082
3387707993828,7cyclictest420-21avahi-daemon00:17:221
3387707993810,23cyclictest1656768-21cups-browsed22:56:061
338770699383,29cyclictest3492237-21ssh23:01:100
3387706993831,4cyclictest3431429-21ls21:21:210
3387708993729,5cyclictest694-21cron22:15:002
3387708993727,6cyclictest3519882-21mailstats23:46:162
3387708993727,5cyclictest3384992-21kworker/u8:11-events_unbound19:21:102
3387708993724,9cyclictest3387705-21cyclictest22:16:032
338770899372,21cyclictest3434143-21sshd21:26:072
3387707993729,6cyclictest3509216-21/usr/sbin/munin23:30:011
3387707993729,5cyclictest421-21dbus-daemon00:10:011
3387707993729,5cyclictest3478779-21latency_hist22:40:021
3387707993728,5cyclictest3424545-21gltestperf21:10:591
3387707993724,9cyclictest421-21dbus-daemon21:39:431
3387706993729,5cyclictest3404135-21unixbench_singl20:01:190
338770699372,27cyclictest3440227-21ssh21:36:050
3387708993628,4cyclictest0-21swapper/200:20:572
3387708993627,6cyclictest623-21sshd23:17:172
3387708993627,4cyclictest0-21swapper/222:30:022
3387708993626,6cyclictest1656763-21cupsd21:10:192
3387708993626,4cyclictest0-21swapper/220:40:002
3387708993625,7cyclictest420-21avahi-daemon21:16:132
3387707993629,4cyclictest3518379-21sh23:45:011
3387707993628,5cyclictest3398227-21systemd-run19:45:011
3387707993627,7cyclictest3419996-21needreboot20:56:011
3387706993628,5cyclictest151ktimers/019:21:040
338770699362,28cyclictest665-21snmpd21:46:080
3387706993621,8cyclictest3461646-21ssh22:11:140
3387708993525,7cyclictest3451790-21head21:55:552
3387708993525,7cyclictest1656763-21cupsd22:21:032
3387707993526,7cyclictest3500079-21ssh23:15:011
3387707993526,7cyclictest3394508-21needreboot19:31:021
3387707993526,7cyclictest3394508-21needreboot19:31:021
3387707993524,8cyclictest3452848-21timerandwakeup21:56:161
3387706993527,5cyclictest3512284-21ssh23:35:010
3387706993519,9cyclictest12650irq/32-eth021:26:090
338770699351,29cyclictest3528731-21sshd00:01:060
3387708993425,6cyclictest421-21dbus-daemon19:40:012
3387708993425,6cyclictest421-21dbus-daemon19:40:012
3387708993425,6cyclictest3413196-21munin-plugin-st20:35:012
3387708993425,5cyclictest1002-21mta-sts-daemon23:53:342
3387707993429,3cyclictest0-21swapper/100:00:021
3387707993427,4cyclictest3480538-21grep22:41:191
3387707993426,5cyclictest3483422-21timerwakeupswit22:46:171
3387707993426,5cyclictest0-21swapper/122:01:071
3387707993425,6cyclictest420-21avahi-daemon22:50:571
338770799342,25cyclictest3449577-21ssh21:51:161
3387706993424,7cyclictest3471307-21unixbench_multi22:26:170
338770699342,24cyclictest3476872-21sshd22:36:060
3387708993322,8cyclictest3416204-21munin-plugin-st20:45:002
3387708993322,8cyclictest3416204-21munin-plugin-st20:45:002
338770899332,26cyclictest3495248-21ssh23:06:082
338770899332,26cyclictest3495248-21ssh23:06:082
338770899331,25cyclictest3480032-21ssh22:41:112
3387707993326,4cyclictest1003-21mta-sts-daemon23:30:381
3387707993325,5cyclictest3542773-21idleruntime-cro00:25:011
3387707993325,5cyclictest3506161-21cron23:25:011
3387707993324,5cyclictest3492701-21unixbench_multi23:01:181
3387707993324,4cyclictest0-21swapper/120:11:051
338770699338,20cyclictest665-21snmpd22:06:070
3387706993328,3cyclictest0-21swapper/019:51:050
3387706993327,4cyclictest3393722-21systemd-run19:30:010
3387706993327,2cyclictest0-21swapper/023:00:030
3387706993325,5cyclictest265-21systemd-journal21:20:010
3387706993324,7cyclictest1002-21mta-sts-daemon20:46:170
3387706993321,9cyclictest0-21swapper/021:51:050
3387706993320,11cyclictest3457385-21systemd-run22:05:020
3387706993317,12cyclictest12650irq/32-eth023:21:100
3387708993226,3cyclictest0-21swapper/200:00:022
3387708993225,4cyclictest3446984-21unixbench_singl21:46:192
3387708993225,3cyclictest0-21swapper/221:41:102
3387708993222,4cyclictest0-21swapper/200:04:212
3387708993215,10cyclictest567-21gdbus20:01:082
3387707993226,3cyclictest0-21swapper/123:05:551
3387707993226,3cyclictest0-21swapper/123:05:551
3387707993224,5cyclictest3429872-21ssh21:20:021
3387707993224,5cyclictest1755695-21munin-node22:17:481
3387707993224,5cyclictest1656763-21cupsd21:21:421
3387707993222,7cyclictest3417099-21ntp_kernel_pll_20:46:071
338770799321,25cyclictest3513465-21ssh23:36:051
3387706993225,5cyclictest3503088-21cron23:20:000
3387706993225,4cyclictest1002-21mta-sts-daemon00:33:580
3387706993222,8cyclictest3423596-21unixbench_multi21:06:170
3387706993218,11cyclictest12650irq/32-eth023:36:050
3387708993122,5cyclictest0-21swapper/221:51:002
338770899311,24cyclictest3440275-21ssh21:36:072
3387707993124,4cyclictest3522864-21sendmail_mailst23:51:191
3387707993123,5cyclictest3546376-21aten_r2power_cu00:30:551
3387707993123,5cyclictest3473217-21aten_r2power_po22:30:551
3387707993122,6cyclictest1000-21mta-sts-daemon00:26:061
3387706993126,3cyclictest3413078-21unixbench_multi20:31:160
3387706993124,5cyclictest0-21swapper/020:20:010
3387706993124,4cyclictest3533614-21munin-plugin-st00:10:010
3387706993124,4cyclictest3402814-21latency_hist20:00:020
3387706993124,4cyclictest151ktimers/020:21:040
3387706993121,7cyclictest0-21swapper/023:11:040
338770699311,25cyclictest3544036-21rm00:26:080
3387708993024,3cyclictest3390736-21systemd-run19:20:022
3387708993024,3cyclictest0-21swapper/220:21:062
3387708993023,3cyclictest0-21swapper/200:06:142
3387708993022,5cyclictest3393585-21unixbench_multi19:26:172
3387708993022,4cyclictest1000-21mta-sts-daemon00:30:122
3387708993019,8cyclictest694-21cron19:45:002
3387708993019,6cyclictest3399711-21systemd-run19:50:022
3387707993023,4cyclictest3404102-21unixbench_multi20:01:191
3387707993021,6cyclictest3470224-21aten2.4-expect22:25:551
3387706993026,2cyclictest0-21swapper/022:00:010
3387706993025,2cyclictest0-21swapper/000:15:010
3387706993024,3cyclictest0-21swapper/019:16:050
3387706993023,4cyclictest3473396-21fschecks_count22:30:580
338770699301,21cyclictest665-21snmpd20:36:060
3387708992923,2cyclictest0-21swapper/220:00:012
3387708992922,3cyclictest0-21swapper/220:30:012
3387708992920,5cyclictest3408216-21proc_pri20:16:112
3387708992917,3cyclictest3507392-21ntpq23:26:072
3387707992922,3cyclictest0-21swapper/121:31:061
3387707992921,5cyclictest1-21systemd22:10:011
338770799291,25cyclictest3422942-21grep21:06:021
3387706992923,4cyclictest3469547-21idleruntime-cro22:25:000
3387706992922,5cyclictest421-21dbus-daemon23:45:010
3387706992921,5cyclictest3464131-21cpu22:15:550
3387708992821,3cyclictest0-21swapper/222:01:142
3387708992819,3cyclictest171rcu_preempt20:46:102
338770799283,12cyclictest3423697-21diskmemload21:41:141
3387707992823,3cyclictest0-21swapper/121:45:561
3387707992821,4cyclictest3423697-21diskmemload23:50:031
3387707992821,4cyclictest3410062-21unixbench-2d20:21:161
3387707992821,4cyclictest3392125-21unixbench_singl19:21:191
3387707992820,5cyclictest3405631-21sed20:06:211
3387706992823,3cyclictest3523243-21grep23:51:240
3387706992821,4cyclictest0-21swapper/021:11:090
3387706992818,8cyclictest3395185-21cron19:35:000
3387706992818,8cyclictest3395185-21cron19:35:000
338770899275,3cyclictest171rcu_preempt20:51:102
3387708992722,3cyclictest0-21swapper/221:00:012
3387708992718,6cyclictest1656768-21cups-browsed20:11:362
3387707992721,3cyclictest0-21swapper/120:40:001
3387707992720,4cyclictest0-21swapper/120:45:001
3387707992720,4cyclictest0-21swapper/120:45:001
3387707992719,5cyclictest420-21avahi-daemon19:27:121
3387706992721,4cyclictest3418433-21missed_timers20:51:020
3387706992721,4cyclictest3399597-21unixbench_multi19:46:200
3387706992720,4cyclictest0-21swapper/000:23:260
338770699271,21cyclictest3396049-21sshd19:36:050
338770699271,21cyclictest3396049-21sshd19:36:050
338770899268,3cyclictest171rcu_preempt21:06:052
3387708992620,3cyclictest1002-21mta-sts-daemon19:10:582
338770799262,18cyclictest3411123-21sshd20:26:051
3387707992617,5cyclictest1656768-21cups-browsed19:17:181
3387706992621,2cyclictest0-21swapper/021:40:590
3387706992619,5cyclictest420-21avahi-daemon20:41:090
3387706992619,5cyclictest420-21avahi-daemon20:41:090
3387706992619,5cyclictest0-21swapper/020:15:020
3387706992619,4cyclictest1656768-21cups-browsed20:59:290
3387706992619,4cyclictest151ktimers/019:41:160
338770799252,20cyclictest3408080-21sshd20:16:051
3387707992517,5cyclictest1656768-21cups-browsed20:32:081
3387707992517,5cyclictest1656768-21cups-browsed19:56:331
3387706992514,8cyclictest0-21swapper/020:06:210
3387708992418,3cyclictest1002-21mta-sts-daemon20:05:482
3387708992417,3cyclictest0-21swapper/219:50:312
3387707992418,4cyclictest3387705-21cyclictest19:12:091
3387707992417,4cyclictest3422195-21cron21:05:011
3387706992316,4cyclictest3411629-21unixbench_singl20:26:150
338770799222,15cyclictest3396150-21ntp_states19:36:111
338770799222,15cyclictest3396150-21ntp_states19:36:111
3387707992213,6cyclictest3399152-21grep19:46:111
338770699222,15cyclictest3421663-21ntp_states21:01:120
3387706992216,4cyclictest3388157-21cpuspeed19:10:580
338770799212,15cyclictest1001-21mta-sts-daemon20:51:071
338770899207,3cyclictest171rcu_preempt19:10:012
3387707992011,6cyclictest3387772-21sort19:10:021
338770999182,8cyclictest0-21swapper/300:06:093
3387706991814,2cyclictest0-21swapper/019:10:010
338770999173,7cyclictest0-21swapper/323:06:193
338770999173,7cyclictest0-21swapper/323:06:193
338770999172,7cyclictest0-21swapper/322:06:073
338770999164,6cyclictest0-21swapper/322:16:073
338770999163,7cyclictest0-21swapper/320:00:013
338770999162,8cyclictest0-21swapper/322:21:043
338770999162,7cyclictest0-21swapper/323:30:583
338770999161,9cyclictest0-21swapper/323:46:063
338770999161,8cyclictest0-21swapper/322:41:053
338770999161,8cyclictest0-21swapper/321:41:103
338770999161,7cyclictest0-21swapper/321:56:043
338770999153,8cyclictest0-21swapper/323:54:073
338770999153,7cyclictest0-21swapper/322:35:593
338770999153,7cyclictest0-21swapper/321:11:053
338770999153,7cyclictest0-21swapper/300:11:193
338770999152,8cyclictest0-21swapper/321:36:093
338770999152,8cyclictest0-21swapper/300:30:023
338770999152,7cyclictest0-21swapper/323:56:053
338770999152,7cyclictest0-21swapper/323:21:123
338770999152,7cyclictest0-21swapper/322:31:083
338770999152,7cyclictest0-21swapper/322:26:043
338770999152,7cyclictest0-21swapper/321:01:073
338770999152,6cyclictest0-21swapper/319:16:063
338770999152,5cyclictest0-21swapper/319:46:103
338770999151,9cyclictest0-21swapper/322:46:073
338770999151,9cyclictest0-21swapper/322:01:123
338770999151,6cyclictest0-21swapper/320:26:053
338770999143,6cyclictest0-21swapper/319:31:043
338770999143,6cyclictest0-21swapper/319:31:043
338770999142,6cyclictest0-21swapper/320:21:033
338770999141,8cyclictest0-21swapper/322:56:013
338770999141,8cyclictest0-21swapper/322:11:183
338770999141,8cyclictest0-21swapper/321:51:163
338770999141,8cyclictest0-21swapper/321:16:113
338770999141,8cyclictest0-21swapper/300:36:073
338770999141,8cyclictest0-21swapper/300:36:073
338770999141,8cyclictest0-21swapper/300:01:073
338770999141,7cyclictest0-21swapper/300:16:073
338770999141,6cyclictest0-21swapper/320:06:093
338770999133,6cyclictest0-21swapper/323:36:073
338770999133,6cyclictest0-21swapper/321:21:043
338770999132,8cyclictest0-21swapper/323:11:253
338770999132,7cyclictest0-21swapper/300:21:043
338770999131,8cyclictest0-21swapper/323:41:123
338770999131,7cyclictest0-21swapper/321:31:003
338770999131,7cyclictest0-21swapper/321:25:593
338770999131,7cyclictest0-21swapper/319:41:153
338770999131,6cyclictest0-21swapper/321:06:063
338770999131,6cyclictest0-21swapper/320:36:053
338770999131,6cyclictest0-21swapper/319:21:053
338770999124,6cyclictest0-21swapper/320:51:103
338770999124,6cyclictest0-21swapper/300:30:363
338770999123,6cyclictest0-21swapper/319:26:053
338770999122,6cyclictest0-21swapper/323:25:573
338770999121,7cyclictest0-21swapper/319:50:593
338770999121,6cyclictest0-21swapper/321:50:023
338770999121,6cyclictest0-21swapper/320:56:033
338770999121,6cyclictest0-21swapper/319:36:053
338770999121,6cyclictest0-21swapper/319:36:053
338770999121,5cyclictest0-21swapper/323:15:533
338770999121,5cyclictest0-21swapper/320:46:043
338770999121,5cyclictest0-21swapper/319:11:073
338770999113,6cyclictest0-21swapper/322:50:553
338770999111,6cyclictest0-21swapper/323:00:543
338770999111,6cyclictest0-21swapper/320:41:073
338770999111,6cyclictest0-21swapper/320:41:073
338770999111,6cyclictest0-21swapper/320:35:013
338770999111,6cyclictest0-21swapper/320:11:043
338770999111,5cyclictest0-21swapper/320:15:573
338770999102,5cyclictest0-21swapper/320:00:583
33877099982,3cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional