You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 16:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Tue May 19, 2026 12:43:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
549753995342,4cyclictest0-21swapper/109:15:551
549752995236,9cyclictest12350irq/32-eth010:45:540
549752995231,13cyclictest12350irq/32-eth012:10:570
549753995139,8cyclictest3888198-21cups-browsed09:30:551
549752994938,7cyclictest0-21swapper/011:01:040
549752994938,7cyclictest0-21swapper/011:01:040
549754994737,6cyclictest720709-21grep11:56:122
549754994538,3cyclictest0-21swapper/210:00:002
549753994533,9cyclictest753975-21sendmail12:40:011
549753994533,9cyclictest753975-21sendmail12:40:011
549752994537,6cyclictest1017-21sendmail-mta10:25:530
549753994436,5cyclictest678822-21systemd-run11:05:021
549753994436,5cyclictest678822-21systemd-run11:05:021
549754994333,6cyclictest639512-21fschecks_time10:15:492
549754994332,7cyclictest420-21avahi-daemon10:30:432
549752994337,4cyclictest642568-21systemd-run10:20:010
549752994333,8cyclictest3888198-21cups-browsed09:20:020
549754994232,7cyclictest663448-21/usr/sbin/munin10:45:462
549754994232,7cyclictest643976-21ntpq10:20:592
549754994230,8cyclictest0-21swapper/209:35:442
54975399423,34cyclictest587202-21ssh09:11:021
54975399423,34cyclictest587202-21ssh09:11:021
54975399422,31cyclictest680123-21ssh11:05:571
54975399422,31cyclictest680123-21ssh11:05:571
549752994234,5cyclictest748120-21sed12:31:080
549752994232,7cyclictest1-21systemd11:30:010
549752994222,14cyclictest12350irq/32-eth012:01:030
549753994133,5cyclictest742363-21munin-plugin-st12:25:011
549753994133,5cyclictest718833-21latency_hist11:55:021
54975399413,29cyclictest735945-21ssh12:16:031
549753994131,7cyclictest686809-21systemd-run11:15:011
549752994134,4cyclictest658620-21munin-run10:40:010
549752994127,12cyclictest421-21dbus-daemon09:15:010
549752994127,12cyclictest421-21dbus-daemon09:15:010
549752994127,12cyclictest1-21systemd12:20:020
54975299411,12cyclictest751394-21ssh12:35:560
54975299411,12cyclictest751394-21ssh12:35:560
549754994032,5cyclictest753975-21sendmail12:40:012
549754994032,5cyclictest753975-21sendmail12:40:012
54975399403,4cyclictest708346-21ssh11:41:031
549753994031,6cyclictest421-21dbus-daemon10:05:011
549753994030,6cyclictest1051-21mta-sts-daemon11:45:541
549753994030,5cyclictest0-21swapper/112:25:531
549753994028,9cyclictest663261-21/usr/sbin/munin10:45:471
549753994018,4cyclictest171rcu_preempt08:24:201
549753994018,4cyclictest171rcu_preempt08:24:201
549752994033,4cyclictest702910-21latency_hist11:35:020
54975299403,30cyclictest680157-21ssh11:05:580
54975299403,30cyclictest680157-21ssh11:05:580
549752994031,6cyclictest560255-21systemd-run07:45:010
549754993931,5cyclictest566246-21grep08:05:012
549754993931,5cyclictest421-21dbus-daemon10:55:012
549753993933,3cyclictest0-21swapper/109:20:541
549753993931,5cyclictest748085-21unixbench-2d12:31:081
549753993931,5cyclictest674844-21latency_hist11:00:021
54975399392,31cyclictest696278-21ssh11:26:011
549753993913,11cyclictest28-21ksoftirqd/110:36:021
549752993933,4cyclictest421-21dbus-daemon11:45:010
549752993932,5cyclictest421-21dbus-daemon11:50:010
549752993929,7cyclictest439-21systemd-logind12:25:020
549752993920,13cyclictest12350irq/32-eth011:16:040
549754993833,2cyclictest0-21swapper/212:35:012
549754993828,7cyclictest421-21dbus-daemon10:45:012
549754993826,9cyclictest731354-21aten_r2power_po12:10:462
549753993828,7cyclictest261-21systemd-journal09:38:261
549752993831,4cyclictest1051-21mta-sts-daemon10:58:080
549752993829,6cyclictest607867-21timerwakeupswit09:36:060
549752993824,10cyclictest421-21dbus-daemon10:05:560
549754993730,4cyclictest1055-21mta-sts-daemon12:00:592
549754993729,5cyclictest682775-21cron11:10:002
549754993729,5cyclictest682775-21cron11:10:002
549754993729,5cyclictest599919-21/usr/sbin/munin09:26:082
549754993729,5cyclictest570774-21systemd-run08:20:012
549754993729,5cyclictest3888198-21cups-browsed11:30:522
549754993728,5cyclictest0-21swapper/212:05:562
549754993723,6cyclictest1051-21mta-sts-daemon12:16:062
549753993727,6cyclictest3888198-21cups-browsed10:32:101
54975299374,29cyclictest668038-21ssh10:50:560
54975499363,24cyclictest587038-21ssh09:10:562
54975499363,24cyclictest587038-21ssh09:10:562
549754993628,4cyclictest0-21swapper/212:20:592
549754993626,6cyclictest3888198-21cups-browsed11:22:042
549753993631,2cyclictest0-21swapper/110:55:011
549753993630,4cyclictest549751-21cyclictest12:02:401
549753993629,3cyclictest0-21swapper/109:45:591
549753993629,3cyclictest0-21swapper/109:45:591
549753993627,5cyclictest640550-21sed10:16:081
549753993618,11cyclictest585726-21diskmemload11:55:591
549752993627,6cyclictest261-21systemd-journal09:32:260
549752993624,9cyclictest0-21swapper/011:36:050
549754993529,3cyclictest0-21swapper/209:00:012
549754993528,3cyclictest0-21swapper/208:15:012
549754993527,5cyclictest714791-21systemd-run11:50:012
549754993527,5cyclictest648480-21grep10:26:072
549754993526,6cyclictest1051-21mta-sts-daemon08:31:052
549754993524,8cyclictest591561-21switchtime09:16:072
54975499351,27cyclictest611547-21ssh09:40:582
54975499351,27cyclictest611547-21ssh09:40:582
549753993525,8cyclictest568544-21needreboot08:10:551
549752993528,5cyclictest582721-21basename09:00:010
549752993528,5cyclictest1051-21mta-sts-daemon07:51:260
549752993527,5cyclictest1055-21mta-sts-daemon11:11:030
549752993525,7cyclictest611825-21switchtime09:41:050
549752993525,7cyclictest611825-21switchtime09:41:050
549754993426,5cyclictest620201-21tail09:51:072
549754993426,5cyclictest0-21swapper/209:35:012
549754993426,3cyclictest0-21swapper/210:35:592
549754993424,7cyclictest261-21systemd-journal11:40:012
549753993427,4cyclictest0-21swapper/110:40:361
549753993427,4cyclictest0-21swapper/109:50:511
549753993426,5cyclictest732570-21unixbench_singl12:11:071
549753993426,5cyclictest687824-21if_wlan011:15:521
549753993426,5cyclictest611010-21ssh09:40:471
549753993426,5cyclictest611010-21ssh09:40:471
549753993426,5cyclictest557248-21kworker/u8:2-xprtiod08:00:581
549753993426,4cyclictest0-21swapper/110:10:531
549753993423,8cyclictest599638-21switchtime09:26:061
549753993423,7cyclictest0-21swapper/111:21:001
549752993426,5cyclictest578253-21systemd-run08:45:010
549752993425,7cyclictest616122-21unixbench_multi09:46:070
549752993425,7cyclictest616122-21unixbench_multi09:46:070
54975499332,26cyclictest676273-21ssh11:01:032
54975499332,26cyclictest676273-21ssh11:01:032
549753993328,2cyclictest0-21swapper/107:40:011
549753993326,5cyclictest0-21swapper/108:40:011
549753993326,3cyclictest0-21swapper/112:10:001
549753993325,5cyclictest564716-21cron08:00:011
549753993325,4cyclictest0-21swapper/110:23:571
549753993324,6cyclictest563220-21cron07:55:011
54975399331,13cyclictest28-21ksoftirqd/109:56:001
549752993327,3cyclictest0-21swapper/009:56:070
549752993324,7cyclictest556605-21ntp_kernel_pll_07:30:540
549752993324,6cyclictest691254-21sh11:20:260
549752993324,6cyclictest628207-21timerandwakeup10:01:070
549752993323,7cyclictest651752-21missed_timers10:30:500
549754993225,4cyclictest558647-21unixbench_singl07:36:112
549754993223,5cyclictest716648-21unixbench-2d11:51:062
54975499322,23cyclictest659-21snmpd08:25:582
549754993220,9cyclictest615766-21mailstats09:46:032
549754993220,9cyclictest615766-21mailstats09:46:032
549753993225,4cyclictest560310-21latency_hist07:45:021
549753993225,4cyclictest557222-21cron07:35:011
549753993224,5cyclictest647140-21/usr/sbin/munin10:26:061
549753993224,3cyclictest0-21swapper/111:30:581
549753993222,8cyclictest704390-21mailstats11:36:021
549753993216,13cyclictest271ktimers/110:05:561
54975299326,14cyclictest743707-21ssh12:25:560
549752993226,4cyclictest719357-21/usr/sbin/munin11:56:000
549752993223,7cyclictest567739-21idleruntime-cro08:10:000
549752993222,8cyclictest421-21dbus-daemon09:55:010
549752993221,7cyclictest12350irq/32-eth007:25:540
549754993125,3cyclictest0-21swapper/211:00:012
549754993123,5cyclictest628332-21sed10:01:082
549754993123,5cyclictest560241-21grep07:45:012
549754993121,7cyclictest707443-21aten_r2power_cu11:40:452
549753993123,5cyclictest555743-21munin-plugin-st07:30:011
549753993121,7cyclictest1056-21mta-sts-daemon08:45:451
549752993124,4cyclictest585767-21latency_hist09:10:020
549752993124,4cyclictest585767-21latency_hist09:10:020
549752993123,4cyclictest151ktimers/008:26:000
549752993122,6cyclictest572264-21true08:25:010
549752993122,6cyclictest572264-21true08:25:010
54975299312,25cyclictest599403-21ssh09:26:010
54975299311,26cyclictest716235-21ssh11:50:590
549754993024,3cyclictest0-21swapper/211:20:002
549754993023,4cyclictest551168-21unixbench_singl07:11:062
549754993023,3cyclictest0-21swapper/211:25:592
549754993022,4cyclictest0-21swapper/210:10:552
549754993022,4cyclictest0-21swapper/210:05:462
54975499302,23cyclictest684160-21ssh11:11:002
549753993024,3cyclictest0-21swapper/109:06:051
549753993024,3cyclictest0-21swapper/109:06:051
549753993016,2cyclictest171rcu_preempt07:15:541
549752993022,5cyclictest630477-21kworker/u8:11+xprtiod10:10:430
549752993021,6cyclictest3888198-21cups-browsed12:06:170
549752993020,7cyclictest579135-21ntp_kernel_pll_08:45:550
54975499297,19cyclictest743625-21ntpq12:25:532
549754992924,2cyclictest0-21swapper/208:40:012
549754992920,6cyclictest577269-21fschecks_time08:40:482
549754992919,5cyclictest0-21swapper/209:10:012
549754992919,5cyclictest0-21swapper/209:10:012
54975499291,22cyclictest595197-21ssh09:20:532
54975399297,3cyclictest171rcu_preempt08:40:561
549753992914,2cyclictest171rcu_preempt09:01:011
549753992913,4cyclictest171rcu_preempt07:20:551
54975399291,14cyclictest28-21ksoftirqd/107:10:561
549752992924,3cyclictest1051-21mta-sts-daemon09:00:580
549752992922,4cyclictest566266-21systemd-run08:05:010
549752992914,12cyclictest564722-21ls08:00:010
54975299291,24cyclictest558130-21sed07:35:590
549754992823,2cyclictest0-21swapper/207:25:002
549753992821,5cyclictest580426-21latency08:50:501
549753992820,5cyclictest0-21swapper/108:20:001
54975299282,22cyclictest643831-21ssh10:20:530
54975299282,13cyclictest660052-21ntp_offset10:40:580
549752992818,7cyclictest595225-21sh09:20:550
549754992722,2cyclictest0-21swapper/207:20:002
549754992717,7cyclictest695-21cron07:35:012
54975399276,5cyclictest574707-21ntpq08:30:591
549753992715,9cyclictest561107-21ntp_kernel_pll_07:45:581
549752992716,5cyclictest552944-21latency_hist07:20:020
549752992712,11cyclictest421-21dbus-daemon08:35:590
549754992621,3cyclictest1051-21mta-sts-daemon07:55:472
549753992619,3cyclictest0-21swapper/109:00:011
54975399261,8cyclictest28-21ksoftirqd/108:05:561
549753992617,5cyclictest573627-21tail08:26:081
54975499252,13cyclictest562613-21ntpq07:50:562
549754992517,5cyclictest570980-21acpi08:20:442
549754992517,5cyclictest570980-21acpi08:20:442
54975299250,21cyclictest550629-21ntp_kernel_pll_07:10:550
549754992418,4cyclictest62847-21kworker/u8:6-xprtiod07:45:542
549754992417,4cyclictest578262-21kworker/u8:10+rpciod08:50:012
549754992415,6cyclictest1-21systemd09:05:022
549752992418,3cyclictest151ktimers/007:20:560
549752992413,8cyclictest560225-21kworker/u8:8+xprtiod07:45:470
549754992318,2cyclictest0-21swapper/208:55:022
54975299232,16cyclictest574623-21ntpq08:30:550
549752992319,2cyclictest0-21swapper/008:20:010
549754992215,4cyclictest0-21swapper/207:30:022
549753992212,3cyclictest171rcu_preempt07:10:021
549753992212,3cyclictest171rcu_preempt07:10:021
549752992218,2cyclictest0-21swapper/008:55:010
549752992216,4cyclictest620-21sshd08:10:260
54975299218,10cyclictest421-21dbus-daemon07:10:020
54975299218,10cyclictest421-21dbus-daemon07:10:020
549754992015,2cyclictest0-21swapper/208:10:002
54975599181,8cyclictest0-21swapper/309:25:563
54975599173,7cyclictest0-21swapper/309:45:553
54975599173,7cyclictest0-21swapper/309:45:553
54975599171,8cyclictest0-21swapper/310:05:533
549754991712,3cyclictest0-21swapper/207:10:032
549754991712,3cyclictest0-21swapper/207:10:032
54975599162,7cyclictest0-21swapper/310:10:583
54975599161,9cyclictest0-21swapper/310:45:563
54975599161,8cyclictest0-21swapper/312:31:003
54975599152,8cyclictest0-21swapper/309:00:593
54975599152,7cyclictest0-21swapper/309:30:573
54975599151,8cyclictest0-21swapper/311:05:583
54975599151,8cyclictest0-21swapper/311:05:583
54975599151,8cyclictest0-21swapper/309:55:553
54975599143,5cyclictest0-21swapper/312:25:543
54975599143,5cyclictest0-21swapper/311:25:593
54975599143,5cyclictest0-21swapper/311:16:003
54975599142,8cyclictest0-21swapper/310:36:023
54975599142,7cyclictest0-21swapper/312:00:553
54975599142,7cyclictest0-21swapper/309:40:553
54975599142,7cyclictest0-21swapper/309:40:553
54975599142,7cyclictest0-21swapper/308:20:563
54975599142,7cyclictest0-21swapper/308:20:563
54975599142,4cyclictest0-21swapper/308:50:533
54975599141,9cyclictest0-21swapper/310:40:593
54975599141,8cyclictest0-21swapper/311:20:563
54975599141,8cyclictest0-21swapper/309:16:003
54975599141,7cyclictest0-21swapper/310:50:553
54975599141,7cyclictest0-21swapper/309:20:543
54975599141,6cyclictest0-21swapper/307:30:573
54975599141,6cyclictest0-21swapper/307:20:593
54975599134,7cyclictest0-21swapper/309:50:543
54975599133,7cyclictest0-21swapper/309:35:563
54975599133,5cyclictest0-21swapper/308:56:023
54975599132,7cyclictest0-21swapper/310:26:013
54975599132,7cyclictest0-21swapper/310:20:593
54975599131,8cyclictest0-21swapper/311:50:573
54975599131,8cyclictest0-21swapper/311:45:013
54975599131,8cyclictest0-21swapper/311:30:543
54975599131,7cyclictest0-21swapper/311:55:583
54975599131,7cyclictest0-21swapper/311:00:583
54975599131,7cyclictest0-21swapper/311:00:583
54975599131,7cyclictest0-21swapper/310:30:543
54975599131,7cyclictest0-21swapper/310:19:313
54975599131,7cyclictest0-21swapper/309:10:573
54975599131,7cyclictest0-21swapper/309:10:573
54975599131,6cyclictest0-21swapper/308:45:543
54975599131,6cyclictest0-21swapper/308:30:523
54975599131,6cyclictest0-21swapper/308:15:583
54975599131,6cyclictest0-21swapper/307:40:553
54975599131,5cyclictest0-21swapper/308:25:573
54975599131,5cyclictest0-21swapper/307:35:563
54975599122,5cyclictest0-21swapper/308:36:003
54975599121,7cyclictest0-21swapper/310:00:573
54975599121,7cyclictest0-21swapper/308:00:533
54975599121,7cyclictest0-21swapper/307:45:573
54975599121,6cyclictest0-21swapper/311:45:523
54975599121,6cyclictest0-21swapper/308:05:563
54975599121,6cyclictest0-21swapper/307:50:543
54975599121,5cyclictest0-21swapper/308:40:563
54975599121,5cyclictest0-21swapper/308:10:563
54975599121,5cyclictest0-21swapper/307:25:543
54975599120,7cyclictest0-21swapper/312:05:543
54975599112,6cyclictest0-21swapper/312:10:533
54975599112,5cyclictest0-21swapper/310:55:463
54975599111,7cyclictest0-21swapper/311:35:523
54975599111,6cyclictest0-21swapper/312:20:483
54975599111,6cyclictest0-21swapper/312:15:503
54975599111,6cyclictest0-21swapper/309:06:023
54975599111,6cyclictest0-21swapper/309:06:023
54975599111,6cyclictest0-21swapper/307:55:533
54975599101,6cyclictest0-21swapper/312:35:483
54975599101,6cyclictest0-21swapper/312:35:483
54975599101,6cyclictest0-21swapper/307:10:573
5497559991,5cyclictest0-21swapper/307:15:463
5497559991,4cyclictest0-21swapper/311:11:033
5497559971,5cyclictest0-21swapper/307:10:013
5497559971,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional