You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 12:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Aug 13, 2026 00:43:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
786313996048,8cyclictest907980-21cron23:30:011
786313995646,6cyclictest898847-21latency_hist23:15:011
786313995646,6cyclictest898847-21latency_hist23:15:011
786313995239,9cyclictest845006-21ssh21:45:541
786313995039,7cyclictest420-21avahi-daemon22:45:141
786312994817,9cyclictest0-21swapper/023:40:550
786312994730,11cyclictest12650irq/32-eth023:20:560
78631499465,32cyclictest857253-21ssh22:05:562
786313994533,6cyclictest0-21swapper/122:25:011
786313994533,6cyclictest0-21swapper/122:25:011
786314994436,5cyclictest823078-21sshd21:10:512
786314994335,4cyclictest0-21swapper/220:20:002
78631499433,35cyclictest863429-21ssh22:15:562
786313994332,7cyclictest3284324-21cups-browsed22:52:231
786313994234,5cyclictest874397-21cron22:35:011
786313994232,6cyclictest914074-21cron23:40:011
786312994233,6cyclictest420-21avahi-daemon23:14:580
786312994233,6cyclictest420-21avahi-daemon23:14:580
786314994131,6cyclictest421-21dbus-daemon22:41:012
786313994130,7cyclictest902078-21latency_hist23:20:021
786314994033,4cyclictest938622-21kworker/u8:0-rpciod00:20:592
786314994033,3cyclictest0-21swapper/222:25:012
786314994033,3cyclictest0-21swapper/222:25:012
786314994031,6cyclictest3284324-21cups-browsed19:55:472
786313994029,7cyclictest0-21swapper/121:26:001
786313994027,9cyclictest949268-21switchtime00:36:061
786312994033,4cyclictest891244-21timerandwakeup23:01:050
786312994026,11cyclictest935555-21ssh00:15:020
786312994023,14cyclictest799864-21latency_hist19:55:020
786314993931,5cyclictest932462-21ssh00:10:012
786314993930,5cyclictest862170-21ssh22:15:012
786313993934,3cyclictest886000-21kworker/u8:3-xprtiod23:00:011
786312993922,12cyclictest12650irq/32-eth022:20:580
786312993922,12cyclictest12650irq/32-eth022:20:580
786314993825,3cyclictest829801-21ntpq21:21:032
786313993829,6cyclictest796797-21cron19:45:011
786314993731,3cyclictest0-21swapper/200:00:012
786314993730,3cyclictest0-21swapper/222:45:572
786314993730,3cyclictest0-21swapper/221:45:022
786314993729,5cyclictest914087-21munin-run23:40:012
786314993728,6cyclictest265-21systemd-journal22:05:012
786313993731,3cyclictest0-21swapper/119:40:011
786313993729,5cyclictest1002-21mta-sts-daemon00:31:201
786313993729,4cyclictest825361-21idleruntime-cro21:15:011
786313993726,8cyclictest801283-21cron20:00:001
786312993730,5cyclictest3284324-21cups-browsed23:55:560
786312993727,7cyclictest889670-21munin-plugin-st23:00:010
786312993727,6cyclictest936747-21ssh00:15:590
786314993628,5cyclictest884999-21mailstats22:51:022
786314993627,5cyclictest837803-21latency_hist21:35:022
786314993627,5cyclictest837803-21latency_hist21:35:022
786314993625,8cyclictest874856-21/usr/sbin/munin22:36:042
78631399364,25cyclictest841997-21ssh21:40:571
786313993630,3cyclictest0-21swapper/100:15:011
786313993628,5cyclictest1-21systemd23:05:011
786313993627,6cyclictest810834-21fschecks_time20:30:511
786313993626,6cyclictest1-21systemd19:55:011
786313993625,6cyclictest0-21swapper/123:50:581
78631399361,27cyclictest863289-21ssh22:15:511
786312993629,5cyclictest151ktimers/000:05:580
786312993622,11cyclictest877450-21ssh22:40:010
786312993617,12cyclictest864199-21ssh22:16:460
78631299361,30cyclictest832923-21ssh21:26:030
78631499353,27cyclictest897043-21ssh23:11:002
78631499353,27cyclictest897043-21ssh23:11:002
786314993528,3cyclictest0-21swapper/223:51:052
786314993527,5cyclictest819317-21systemd-run21:00:012
786314993527,5cyclictest793802-21munin-plugin-st19:35:012
786314993527,5cyclictest421-21dbus-daemon00:15:022
78631399354,21cyclictest893965-21sshd23:06:021
786313993529,3cyclictest0-21swapper/119:20:011
786313993527,5cyclictest915183-21needreboot23:40:521
786312993527,5cyclictest918845-21unixbench-2d23:46:060
786312993527,5cyclictest420-21avahi-daemon21:53:270
78631499345,24cyclictest915401-21ssh23:41:012
786314993428,3cyclictest0-21swapper/200:19:032
786314993427,4cyclictest899578-21ssh23:15:482
786314993426,4cyclictest0-21swapper/223:06:022
786314993425,6cyclictest421-21dbus-daemon23:05:012
786313993427,4cyclictest838337-21aten_r2power_vo21:35:461
78631399341,5cyclictest171rcu_preempt20:40:541
786312993427,4cyclictest869201-21kernelversion22:25:500
786312993426,5cyclictest945151-21/usr/sbin/munin00:30:460
786312993426,5cyclictest822291-21cron21:10:010
786312993426,5cyclictest3284324-21cups-browsed21:40:350
786312993420,4cyclictest171rcu_preempt22:00:560
786312993417,10cyclictest912161-21ssh23:35:530
78631299341,6cyclictest171rcu_preempt21:30:570
78631299341,6cyclictest171rcu_preempt21:30:570
786314993327,3cyclictest0-21swapper/221:35:492
786314993326,3cyclictest0-21swapper/200:40:032
786314993325,5cyclictest816318-21systemd20:50:022
786314993325,5cyclictest0-21swapper/222:35:012
786314993325,5cyclictest0-21swapper/219:20:012
786313993327,3cyclictest0-21swapper/100:00:021
786313993326,4cyclictest820675-21unixbench_multi21:01:061
786313993326,3cyclictest0-21swapper/121:20:011
786313993325,4cyclictest0-21swapper/119:15:011
786313993324,6cyclictest853005-21cat22:00:011
786313993320,8cyclictest0-21swapper/123:30:541
786312993325,6cyclictest846903-21/usr/sbin/munin21:50:010
786312993325,5cyclictest905387-21/usr/sbin/munin23:25:520
786312993325,5cyclictest793696-21unixbench_multi19:31:070
78631499328,5cyclictest171rcu_preempt00:26:012
786314993225,4cyclictest820795-21cron21:05:012
786314993222,6cyclictest3284321-21cupsd23:30:112
786314993217,11cyclictest421-21dbus-daemon00:05:022
786313993226,3cyclictest0-21swapper/119:30:011
786313993225,4cyclictest1-21systemd21:00:021
786313993224,5cyclictest877543-21sshd22:40:021
786313993223,4cyclictest0-21swapper/100:20:491
786313993222,7cyclictest920210-21cut23:50:021
786313993222,6cyclictest421-21dbus-daemon00:08:591
786313993221,7cyclictest926384-21cups-browsed00:26:031
786313993221,7cyclictest420-21avahi-daemon22:00:371
786312993227,2cyclictest0-21swapper/021:40:000
786312993226,3cyclictest899724-21kernelversion23:15:500
786312993225,4cyclictest908920-21latency23:30:510
786312993220,9cyclictest265-21systemd-journal00:28:000
786312993220,9cyclictest0-21swapper/022:11:060
786312993216,3cyclictest171rcu_preempt22:33:320
786314993126,2cyclictest0-21swapper/220:15:012
786314993126,2cyclictest0-21swapper/220:15:012
786314993123,5cyclictest889807-21latency_hist23:00:022
78631499312,22cyclictest845095-21ssh21:45:582
786314993120,3cyclictest171rcu_preempt20:00:562
786313993124,4cyclictest836405-21ssh21:31:071
786313993124,4cyclictest836405-21ssh21:31:071
786313993123,5cyclictest421-21dbus-daemon20:30:011
786313993123,5cyclictest421-21dbus-daemon20:30:011
786313993123,5cyclictest113797-21kworker/u8:14+xprtiod19:25:011
78631399312,23cyclictest860396-21ssh22:10:591
786312993125,4cyclictest789171-21sed19:16:060
786312993125,3cyclictest885406-21kworker/u8:1-xprtiod23:51:050
786312993124,4cyclictest796691-21unixbench_multi19:41:100
786312993122,7cyclictest815702-21ntp_kernel_pll_20:45:550
786312993122,7cyclictest794611-21needreboot19:35:520
786312993121,7cyclictest0-21swapper/000:00:580
786312993119,6cyclictest881731-21sh22:45:570
78631299311,14cyclictest854327-21kworker/u8:6+rpciod22:05:550
78631499304,20cyclictest694-21cron21:56:002
786314993024,3cyclictest0-21swapper/221:26:092
786314993024,3cyclictest0-21swapper/200:30:572
786314993023,3cyclictest0-21swapper/222:25:532
786314993022,5cyclictest809212-21aten_r2power_vo20:25:462
786314993022,5cyclictest809212-21aten_r2power_vo20:25:462
786314993022,5cyclictest808807-21sh20:25:002
786314993022,5cyclictest808807-21sh20:25:002
786314993020,7cyclictest791682-21ntp_kernel_pll_19:25:562
786314993018,3cyclictest904537-21ssh23:23:472
786314993018,3cyclictest171rcu_preempt20:35:562
78631399303,21cyclictest665-21snmpd20:00:591
786313993023,4cyclictest859171-21grep22:10:021
786313993023,4cyclictest848602-21unixbench_multi21:51:061
786313993023,4cyclictest0-21swapper/122:45:011
786313993023,3cyclictest0-21swapper/119:30:561
786313993022,5cyclictest807129-21unixbench-2d20:16:091
786312993023,3cyclictest885406-21kworker/u8:1-rpciod23:05:590
786312993023,3cyclictest0-21swapper/020:20:560
786312993023,3cyclictest0-21swapper/020:20:560
786312993022,5cyclictest807336-21unin-run20:20:010
786312993020,7cyclictest421-21dbus-daemon20:00:010
786312993014,11cyclictest420-21avahi-daemon22:41:010
78631299301,24cyclictest420-21avahi-daemon20:51:010
786314992922,4cyclictest918311-21ntp_kernel_pll_23:45:542
786314992918,8cyclictest797678-21ntp_kernel_pll_19:45:572
78631399293,20cyclictest66550snmpd19:45:551
786313992924,2cyclictest0-21swapper/100:15:561
786313992922,3cyclictest0-21swapper/122:25:581
786313992921,5cyclictest926384-21cups-browsed00:00:191
786312992924,2cyclictest0-21swapper/021:15:010
786312992923,3cyclictest0-21swapper/021:15:490
786312992922,2cyclictest14-21ksoftirqd/020:05:540
786312992921,5cyclictest948538-21gltestperf00:35:490
786312992920,7cyclictest809326-21fschecks_count20:25:480
786312992920,7cyclictest809326-21fschecks_count20:25:480
786312992917,5cyclictest787287-21ld19:11:000
786312992914,8cyclictest822295-21diskmemload21:20:550
78631499284,18cyclictest826493-21ssh21:15:552
78631499284,12cyclictest848212-21ssh21:51:012
786314992821,3cyclictest0-21swapper/223:25:542
786314992817,3cyclictest171rcu_preempt19:21:002
78631399288,15cyclictest421-21dbus-daemon20:20:561
78631399288,15cyclictest421-21dbus-daemon20:20:561
786313992821,4cyclictest0-21swapper/120:46:041
786313992820,5cyclictest817793-21munin-plugin-st20:55:011
786313992819,7cyclictest822282-21sshd21:09:471
78631299288,5cyclictest12750irq/33-eth020:00:590
786312992822,4cyclictest813287-21munin-run20:40:010
786312992820,6cyclictest805780-21cron20:15:000
786312992820,6cyclictest805780-21cron20:15:000
786314992719,5cyclictest813836-21fschecks_time20:40:472
786314992719,5cyclictest0-21swapper/219:40:012
78631399272,20cyclictest829710-21ntpq21:20:581
786313992719,5cyclictest804829-21fschecks_time20:10:471
786313992719,5cyclictest804829-21fschecks_time20:10:471
786313992719,4cyclictest0-21swapper/123:21:351
78631299278,3cyclictest171rcu_preempt00:20:540
786312992722,2cyclictest0-21swapper/022:50:470
786312992719,6cyclictest851296-21grep21:56:030
786314992619,4cyclictest822322-21systemd-run21:10:012
786314992619,4cyclictest811264-21ntpq20:31:022
786314992618,5cyclictest803325-21grep20:05:472
786314992615,6cyclictest171rcu_preempt19:50:522
78631399269,14cyclictest271ktimers/120:05:571
78631299263,17cyclictest420-21avahi-daemon20:40:560
786312992620,4cyclictest797339-21fschecks_time19:45:500
786312992615,2cyclictest171rcu_preempt19:20:470
786314992519,3cyclictest0-21swapper/219:11:052
786314992516,6cyclictest796822-21systemd-run19:45:012
786314992516,5cyclictest351ktimers/220:50:552
786313992518,5cyclictest0-21swapper/120:35:511
78631299252,19cyclictest665-21snmpd20:31:040
786312992519,4cyclictest786489-21latency_hist19:10:030
78631299249,3cyclictest0-21swapper/019:25:590
786312992417,4cyclictest0-21swapper/020:55:580
786312992413,6cyclictest171rcu_preempt21:00:570
786314992014,3cyclictest0-21swapper/219:10:022
786313992014,3cyclictest0-21swapper/119:10:021
78631599182,6cyclictest0-21swapper/321:50:563
78631599171,9cyclictest0-21swapper/322:41:013
78631599163,7cyclictest0-21swapper/322:55:553
78631599162,7cyclictest0-21swapper/300:20:553
78631599161,8cyclictest0-21swapper/300:29:253
78631599161,8cyclictest0-21swapper/300:16:013
78631599161,6cyclictest0-21swapper/319:25:553
78631599152,8cyclictest0-21swapper/323:35:543
78631599152,7cyclictest0-21swapper/323:15:533
78631599152,7cyclictest0-21swapper/322:45:573
78631599152,7cyclictest0-21swapper/322:35:543
78631599152,7cyclictest0-21swapper/319:20:563
78631599152,6cyclictest0-21swapper/319:15:533
78631599151,8cyclictest0-21swapper/322:15:563
78631599151,7cyclictest0-21swapper/323:10:573
78631599151,7cyclictest0-21swapper/323:10:573
78631599151,7cyclictest0-21swapper/322:05:563
78631599151,6cyclictest0-21swapper/319:40:573
78631599142,7cyclictest0-21swapper/319:36:003
78631599142,5cyclictest0-21swapper/322:25:553
78631599141,8cyclictest0-21swapper/322:30:553
78631599141,7cyclictest0-21swapper/323:05:583
78631599141,7cyclictest0-21swapper/322:50:533
78631599141,6cyclictest0-21swapper/321:11:013
78631599141,6cyclictest0-21swapper/320:20:563
78631599141,6cyclictest0-21swapper/320:20:563
78631599141,6cyclictest0-21swapper/320:05:533
78631599132,6cyclictest0-21swapper/323:50:013
78631599132,6cyclictest0-21swapper/323:20:593
78631599132,6cyclictest0-21swapper/321:40:583
78631599132,6cyclictest0-21swapper/300:10:583
78631599132,5cyclictest0-21swapper/321:30:543
78631599132,5cyclictest0-21swapper/321:30:543
78631599131,7cyclictest0-21swapper/323:50:523
78631599131,7cyclictest0-21swapper/323:31:023
78631599131,7cyclictest0-21swapper/322:01:043
78631599131,7cyclictest0-21swapper/321:20:553
78631599131,7cyclictest0-21swapper/320:00:563
78631599131,6cyclictest0-21swapper/323:55:553
78631599131,6cyclictest0-21swapper/321:55:473
78631599131,6cyclictest0-21swapper/319:55:593
78631599131,6cyclictest0-21swapper/300:36:053
78631599131,5cyclictest0-21swapper/320:45:523
78631599131,5cyclictest0-21swapper/320:35:533
78631599121,7cyclictest0-21swapper/300:01:013
78631599121,6cyclictest0-21swapper/321:05:573
78631599121,6cyclictest0-21swapper/321:01:003
78631599121,6cyclictest0-21swapper/320:31:043
78631599121,6cyclictest0-21swapper/320:16:023
78631599121,6cyclictest0-21swapper/320:10:533
78631599121,6cyclictest0-21swapper/320:10:533
78631599121,6cyclictest0-21swapper/300:05:563
78631599115,4cyclictest0-21swapper/322:20:583
78631599115,4cyclictest0-21swapper/322:20:583
78631599113,5cyclictest0-21swapper/321:15:463
78631599112,6cyclictest0-21swapper/322:10:543
78631599111,6cyclictest0-21swapper/323:25:543
78631599111,6cyclictest0-21swapper/323:00:593
78631599111,6cyclictest0-21swapper/321:25:543
78631599111,6cyclictest0-21swapper/320:50:533
78631599111,6cyclictest0-21swapper/320:40:533
78631599111,6cyclictest0-21swapper/300:30:563
78631599111,5cyclictest0-21swapper/321:35:163
78631599111,5cyclictest0-21swapper/319:45:473
78631599111,5cyclictest0-21swapper/319:10:533
78631599103,5cyclictest0-21swapper/321:45:023
78631599102,6cyclictest0-21swapper/319:10:023
78631599101,6cyclictest0-21swapper/323:40:293
78631599101,5cyclictest0-21swapper/321:00:023
78631599101,5cyclictest0-21swapper/319:50:523
78631599101,5cyclictest0-21swapper/319:30:583
7863159991,5cyclictest0-21swapper/320:25:413
7863159991,5cyclictest0-21swapper/320:25:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional