You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 06:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Aug 12, 2026 00:43:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3144318995646,6cyclictest0-21swapper/121:26:021
3144319995345,5cyclictest3308664-21sh00:40:012
3144319994837,7cyclictest3184870-21switchtime21:16:062
3144317994838,7cyclictest3284254-21munin-run00:00:010
3144318994637,6cyclictest3305837-21latency_hist00:35:021
314431899463,30cyclictest3196922-21ssh21:35:571
3144317994637,4cyclictest0-21swapper/000:36:000
3144319994530,11cyclictest3180303-21diskmemload23:26:102
3144317994529,13cyclictest12650irq/32-eth023:51:000
3144318994436,5cyclictest3287826-21acpi00:05:471
3144317994430,11cyclictest3250706-21idleruntime-cro23:05:010
3144318994335,5cyclictest3166798-21cron20:25:001
3144317994323,5cyclictest171rcu_preempt21:35:550
3144318994232,7cyclictest3232387-21latency_hist22:35:021
3144317994233,6cyclictest3275704-21aten_r2power_fr23:45:480
3144317994230,9cyclictest567-21gdbus21:55:010
3144318994134,4cyclictest3168370-21latency_hist20:30:011
314431799413,34cyclictest151ktimers/000:25:540
314431799413,34cyclictest151ktimers/000:25:540
314431799412,31cyclictest3190744-21ssh21:25:570
3144319994033,4cyclictest3211018-21ssh22:00:012
3144319994033,3cyclictest0-21swapper/222:10:002
3144319994033,3cyclictest0-21swapper/222:10:002
3144319994032,5cyclictest3159325-21kworker/u8:9-xprtiod20:00:012
314431999402,23cyclictest3300661-21ssh00:25:542
314431999402,23cyclictest3300661-21ssh00:25:542
314431899405,30cyclictest665-21snmpd19:21:031
3144318994033,4cyclictest3201856-21sh21:45:001
3144318994032,5cyclictest3171333-21systemd-run20:40:021
3144318994011,7cyclictest28-21ksoftirqd/120:20:001
3144318994011,7cyclictest28-21ksoftirqd/120:20:001
3144317994034,3cyclictest0-21swapper/022:20:010
3144317994033,4cyclictest3293453-21latency_hist00:15:020
3144317994033,4cyclictest0-21swapper/021:31:010
3144317994033,4cyclictest0-21swapper/021:31:010
3144319993931,5cyclictest439-21systemd-logind23:25:012
3144318993932,4cyclictest3277028-21kworker/u8:8-rpciod00:15:491
3144318993930,6cyclictest420-21avahi-daemon20:06:091
3144318993930,6cyclictest0-21swapper/121:50:481
3144319993831,4cyclictest3183383-21sh21:15:012
3144319993828,7cyclictest421-21dbus-daemon21:48:252
3144318993832,3cyclictest0-21swapper/123:20:491
3144318993830,4cyclictest0-21swapper/100:20:501
3144318993827,8cyclictest3251646-21latency23:05:531
3144318993822,2cyclictest171rcu_preempt23:31:001
3144317993832,4cyclictest3232379-21ssh22:35:020
3144317993832,4cyclictest3168315-21systemd-run20:30:010
3144317993824,4cyclictest3174178-21unixbench_multi20:46:070
314431799382,26cyclictest3200000-21ssh21:40:570
3144317993821,10cyclictest151ktimers/022:51:010
3144317993817,16cyclictest12650irq/32-eth023:11:020
314431999373,26cyclictest3161856-21sshd20:06:092
3144319993728,6cyclictest265-21systemd-journal22:40:002
3144319993727,6cyclictest1434212-21cupsd21:25:162
3144318993729,5cyclictest485-21in:imuxsock22:40:001
314431899371,27cyclictest3212262-21ssh22:01:031
314431899371,27cyclictest3212262-21ssh22:01:031
3144317993731,3cyclictest0-21swapper/000:10:010
3144317993720,11cyclictest3259852-21munin-run23:20:010
3144317993720,11cyclictest3259852-21munin-run23:20:010
314431799371,30cyclictest191rcuc/019:36:010
3144317993710,6cyclictest12750irq/33-eth022:45:560
314431999365,18cyclictest3258057-21ssh23:16:002
314431999365,18cyclictest3258057-21ssh23:16:002
3144319993630,3cyclictest0-21swapper/220:35:022
3144319993626,8cyclictest3180320-21sh21:10:012
3144319993626,7cyclictest3284254-21munin-run00:00:012
3144319993626,7cyclictest265-21systemd-journal00:15:012
3144319993626,7cyclictest1003-21mta-sts-daemon22:20:442
3144318993631,2cyclictest0-21swapper/121:20:011
3144318993628,5cyclictest3306576-21latency00:35:511
3144318993628,5cyclictest3223201-21munin-run22:20:011
3144318993627,6cyclictest421-21dbus-daemon00:13:591
3144318993627,4cyclictest0-21swapper/122:45:571
3144318993626,6cyclictest1434245-21cups-browsed23:03:221
314431899362,24cyclictest3193868-21ssh21:30:591
314431899362,24cyclictest3193868-21ssh21:30:591
314431899361,17cyclictest3258048-21ssh23:15:591
314431899361,17cyclictest3258048-21ssh23:15:591
3144317993627,7cyclictest3177313-21idleruntime-cro21:00:010
3144317993621,5cyclictest1001-21mta-sts-daemon21:56:310
3144319993527,5cyclictest3303366-21fschecks_time00:30:482
3144319993525,6cyclictest421-21dbus-daemon21:35:452
3144319993524,7cyclictest3294015-21aten_r2power_po00:15:462
3144319993523,9cyclictest3193683-21missed_timers21:30:532
3144319993523,9cyclictest3193683-21missed_timers21:30:532
314431999351,28cyclictest3251884-21ssh23:05:592
314431899353,26cyclictest3215313-21ssh22:06:001
314431899353,26cyclictest3215313-21ssh22:06:001
3144318993529,3cyclictest0-21swapper/121:25:011
3144318993528,4cyclictest3220144-21/usr/sbin/munin22:15:011
3144318993527,5cyclictest3159299-21systemctl20:00:011
3144318993527,5cyclictest3149275-21cpuspeed19:25:461
3144318993525,6cyclictest421-21dbus-daemon23:10:541
3144318993523,9cyclictest3278599-21/usr/sbin/munin23:50:521
314431899352,28cyclictest3245793-21ssh22:55:571
3144317993528,5cyclictest3166880-21latency_hist20:25:010
3144317993525,3cyclictest3188232-21sed21:21:090
3144317993523,9cyclictest0-21swapper/023:30:000
3144317993522,9cyclictest3266822-21kworker/u8:223:30:490
314431799351,27cyclictest3303927-21wc00:31:020
3144319993428,3cyclictest0-21swapper/222:40:582
3144319993427,5cyclictest3297348-21if_wlan000:20:502
3144319993427,3cyclictest3195729-21kworker/u8:3-rpciod21:43:172
3144319993426,4cyclictest3242293-21sh22:50:492
3144319993426,4cyclictest3185497-21kworker/u8:7-nfsiod21:25:012
3144319993422,9cyclictest3288330-21latency00:05:532
314431899346,15cyclictest3224525-21ssh22:21:031
3144318993429,2cyclictest0-21swapper/122:00:011
3144318993428,3cyclictest0-21swapper/123:55:571
3144318993427,4cyclictest3243147-21timerwakeupswit22:51:071
3144318993427,3cyclictest0-21swapper/121:50:011
3144318993426,5cyclictest3183383-21cron21:15:011
3144318993425,6cyclictest3238454-21sh22:45:001
314431799345,18cyclictest3285602-21ssh00:01:030
3144317993428,3cyclictest0-21swapper/021:15:450
3144317993427,4cyclictest1001-21mta-sts-daemon00:20:570
3144317993425,7cyclictest3226290-21systemd22:25:020
3144317993419,6cyclictest12650irq/32-eth000:15:580
3144317993413,3cyclictest171rcu_preempt22:10:540
3144317993413,16cyclictest3236374-21latency22:40:530
3144319993327,3cyclictest0-21swapper/222:20:012
3144319993325,5cyclictest0-21swapper/221:55:012
3144319993323,7cyclictest0-21swapper/223:30:452
3144318993326,4cyclictest3226274-21kworker/u8:8-rpciod22:26:491
3144318993320,4cyclictest28-21ksoftirqd/123:25:471
3144318993310,5cyclictest28-21ksoftirqd/123:50:021
3144317993324,6cyclictest420-21avahi-daemon23:23:240
3144317993322,8cyclictest3202319-21acpi21:45:430
314431799332,24cyclictest665-21snmpd19:41:010
314431999323,24cyclictest3218245-21ssh22:10:542
3144319993226,3cyclictest0-21swapper/219:15:002
3144319993225,3cyclictest0-21swapper/223:01:012
3144319993224,5cyclictest3247591-21cron23:00:012
314431999322,26cyclictest3273351-21ssh23:41:022
314431899325,8cyclictest28-21ksoftirqd/119:45:571
3144318993224,4cyclictest3146132-21aten_r2power_fr19:15:451
3144318993222,7cyclictest265-21systemd-journal00:25:361
3144318993222,7cyclictest265-21systemd-journal00:25:361
3144317993225,4cyclictest151ktimers/019:45:560
3144317993222,8cyclictest3147294-21cron19:20:010
3144317993219,10cyclictest3246160-21ssh22:56:060
3144317993216,3cyclictest171rcu_preempt20:35:580
3144319993126,2cyclictest0-21swapper/223:35:522
3144319993124,4cyclictest3153344-21systemd19:40:022
3144319993124,4cyclictest1002-21mta-sts-daemon22:29:072
3144319993121,7cyclictest3154083-21missed_timers19:40:512
3144319993121,6cyclictest0-21swapper/200:00:582
314431899313,20cyclictest3160245-21cron20:01:001
3144318993124,4cyclictest1002-21mta-sts-daemon19:11:171
3144318993123,5cyclictest3175901-21latency_hist20:55:021
3144318993123,5cyclictest3153296-21ls19:40:011
3144318993122,4cyclictest0-21swapper/123:38:271
314431899312,21cyclictest3273400-21ssh23:41:051
3144318993113,3cyclictest171rcu_preempt20:30:481
3144317993122,6cyclictest3149350-21fschecks_time19:25:490
3144317993118,5cyclictest3162203-21tail20:06:140
3144317993117,3cyclictest171rcu_preempt22:35:090
3144317993117,3cyclictest0-21swapper/022:05:560
3144317993117,3cyclictest0-21swapper/022:05:560
3144319993022,5cyclictest3230987-21unixbench-2d22:31:072
3144319993020,7cyclictest439-21systemd-logind20:40:022
3144319993019,8cyclictest3160120-21needreboot20:00:552
314431999301,25cyclictest665-21snmpd23:46:042
3144318993018,3cyclictest171rcu_preempt20:10:551
3144318993018,3cyclictest171rcu_preempt20:10:551
3144317993024,4cyclictest3165292-21sh20:20:000
3144317993024,4cyclictest3165292-21sh20:20:000
3144317993022,6cyclictest3148810-21cron19:25:010
314431799302,22cyclictest3227381-21ssh22:25:530
3144317993021,6cyclictest0-21swapper/023:40:010
3144317993021,3cyclictest171rcu_preempt19:30:550
314431999294,21cyclictest3278891-21diskstats23:50:482
3144319992924,2cyclictest0-21swapper/220:55:012
314431999292,21cyclictest3239577-21ntpq22:45:542
314431899297,5cyclictest28-21ksoftirqd/120:45:451
314431899295,6cyclictest28-21ksoftirqd/121:06:001
314431899294,14cyclictest28-21ksoftirqd/100:01:011
3144318992923,3cyclictest0-21swapper/120:45:021
3144317992922,5cyclictest3251145-21/usr/sbin/munin23:06:090
3144317992917,8cyclictest12650irq/32-eth023:41:050
3144317992912,3cyclictest171rcu_preempt22:00:530
3144317992912,3cyclictest171rcu_preempt22:00:530
314431799291,10cyclictest3180291-21kworker/0:1+events_power_efficient21:12:270
3144319992823,2cyclictest0-21swapper/221:00:012
3144319992821,4cyclictest1000-21mta-sts-daemon19:45:472
3144318992822,3cyclictest0-21swapper/119:55:011
314431899282,15cyclictest28-21ksoftirqd/119:30:591
3144318992812,5cyclictest171rcu_preempt19:40:561
3144317992818,7cyclictest0-21swapper/020:35:010
3144317992812,5cyclictest171rcu_preempt21:00:540
314431999272,21cyclictest3212334-21ssh22:01:062
314431999272,21cyclictest3212334-21ssh22:01:062
3144319992720,4cyclictest3157789-21cron19:55:012
3144319992719,5cyclictest1434245-21cups-browsed20:15:172
3144319992719,5cyclictest1434245-21cups-browsed20:15:172
3144318992720,4cyclictest427-21polkitd21:02:531
3144317992717,3cyclictest171rcu_preempt19:15:010
3144317992714,3cyclictest171rcu_preempt19:50:490
3144317992714,2cyclictest171rcu_preempt20:10:550
3144317992714,2cyclictest171rcu_preempt20:10:550
3144317992713,8cyclictest171rcu_preempt19:55:550
3144319992620,3cyclictest0-21swapper/219:20:012
3144319992615,5cyclictest171rcu_preempt23:12:392
3144318992615,3cyclictest171rcu_preempt20:55:521
3144319992520,2cyclictest0-21swapper/219:30:452
3144319992518,5cyclictest3150181-21unixbench_multi19:26:072
3144319992515,7cyclictest3163353-21sendmail_mailqu20:11:042
3144319992515,7cyclictest3163353-21sendmail_mailqu20:11:042
3144319992514,8cyclictest421-21dbus-daemon21:05:012
314431899256,6cyclictest28-21ksoftirqd/119:10:021
314431799252,5cyclictest171rcu_preempt21:06:190
3144317992519,4cyclictest3159542-21acpi20:00:470
3144317992512,3cyclictest171rcu_preempt20:51:010
3144319992416,5cyclictest3172485-21switchtime20:41:032
3144319992415,6cyclictest1434245-21cups-browsed19:20:042
3144319992415,6cyclictest1000-21mta-sts-daemon20:25:042
3144319992316,4cyclictest3166677-21tail20:21:082
314431999231,12cyclictest3171259-21kworker/u8:1+xprtiod20:45:492
314431799239,7cyclictest171rcu_preempt20:40:450
314431799206,3cyclictest171rcu_preempt19:10:030
314432099181,7cyclictest0-21swapper/322:50:553
314432099173,7cyclictest0-21swapper/323:30:533
314432099171,8cyclictest0-21swapper/321:20:593
314432099171,7cyclictest0-21swapper/320:35:593
314432099162,9cyclictest0-21swapper/300:25:563
314432099162,9cyclictest0-21swapper/300:25:563
314432099161,9cyclictest0-21swapper/321:26:003
314432099161,7cyclictest0-21swapper/323:50:563
314431999161,11cyclictest3144325-21systemd-run19:10:022
314432099153,8cyclictest0-21swapper/322:55:593
314432099153,6cyclictest0-21swapper/323:15:543
314432099153,6cyclictest0-21swapper/323:15:543
314432099153,6cyclictest0-21swapper/323:00:593
314432099153,6cyclictest0-21swapper/322:45:533
314432099153,6cyclictest0-21swapper/300:10:583
314432099152,9cyclictest0-21swapper/322:20:573
314432099152,7cyclictest0-21swapper/323:46:083
314432099152,7cyclictest0-21swapper/321:10:563
314432099151,8cyclictest0-21swapper/322:31:033
314432099151,7cyclictest0-21swapper/322:05:583
314432099151,7cyclictest0-21swapper/322:05:583
314432099151,7cyclictest0-21swapper/321:55:553
314432099151,6cyclictest0-21swapper/320:50:543
314432099151,6cyclictest0-21swapper/320:45:563
314432099151,6cyclictest0-21swapper/320:21:003
314432099151,6cyclictest0-21swapper/319:55:583
314432099146,4cyclictest0-21swapper/321:15:543
314432099143,7cyclictest0-21swapper/321:35:563
314432099142,7cyclictest0-21swapper/322:40:593
314432099142,7cyclictest0-21swapper/321:31:003
314432099142,7cyclictest0-21swapper/321:31:003
314432099142,7cyclictest0-21swapper/300:20:583
314432099142,6cyclictest0-21swapper/323:40:563
314432099142,6cyclictest0-21swapper/323:05:583
314432099142,6cyclictest0-21swapper/322:25:523
314432099142,6cyclictest0-21swapper/321:40:533
314432099141,8cyclictest0-21swapper/300:06:043
314432099141,7cyclictest0-21swapper/322:11:003
314432099141,7cyclictest0-21swapper/321:45:573
314432099141,7cyclictest0-21swapper/320:11:033
314432099141,7cyclictest0-21swapper/320:11:033
314432099141,7cyclictest0-21swapper/319:30:533
314432099141,7cyclictest0-21swapper/319:10:573
314432099141,6cyclictest0-21swapper/320:30:543
314432099141,6cyclictest0-21swapper/319:45:533
314432099132,7cyclictest0-21swapper/321:54:183
314432099132,6cyclictest0-21swapper/323:25:583
314432099132,6cyclictest0-21swapper/323:10:573
314432099132,6cyclictest0-21swapper/322:20:013
314432099131,6cyclictest0-21swapper/323:55:463
314432099126,4cyclictest0-21swapper/319:21:033
314432099123,5cyclictest0-21swapper/300:35:583
314432099123,4cyclictest0-21swapper/321:06:043
314432099122,6cyclictest0-21swapper/323:20:493
314432099122,6cyclictest0-21swapper/322:01:023
314432099122,6cyclictest0-21swapper/322:01:023
314432099122,5cyclictest0-21swapper/320:20:003
314432099122,5cyclictest0-21swapper/320:20:003
314432099122,5cyclictest0-21swapper/319:40:553
314432099121,6cyclictest0-21swapper/323:35:523
314432099121,6cyclictest0-21swapper/320:25:553
314432099121,6cyclictest0-21swapper/320:01:003
314432099121,6cyclictest0-21swapper/300:00:523
314432099121,5cyclictest0-21swapper/320:55:543
314432099112,6cyclictest0-21swapper/322:35:453
314432099112,6cyclictest0-21swapper/300:30:593
314432099112,5cyclictest0-21swapper/319:25:533
314432099111,6cyclictest0-21swapper/321:00:553
314432099111,6cyclictest0-21swapper/320:06:093
314432099111,6cyclictest0-21swapper/300:15:363
314432099111,5cyclictest0-21swapper/319:50:523
314432099101,3cyclictest0-21swapper/320:40:513
31443209991,5cyclictest0-21swapper/319:35:453
31443209991,5cyclictest0-21swapper/319:15:453
31443209981,4cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional