You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 20:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat May 23, 2026 12:43:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
66328995342,7cyclictest106495-21latency_hist09:15:031
66328995342,7cyclictest106495-21latency_hist09:15:031
66327995314,11cyclictest151ktimers/012:31:010
66327995314,11cyclictest151ktimers/012:31:010
66328995239,7cyclictest141667-21kworker/u8:0-rpciod10:25:591
66327994921,22cyclictest251763-21kworker/u8:13+rpciod12:26:060
66327994833,12cyclictest91880-21latency_hist08:35:020
66329994720,22cyclictest538-21wpa_supplicant07:41:002
66329994720,22cyclictest538-21wpa_supplicant07:41:002
66328994721,6cyclictest155240-21kworker/u8:3-xprtiod10:51:031
66329994636,6cyclictest239772-21latency_hist12:00:022
66328994536,3cyclictest0-21swapper/109:26:021
66327994535,7cyclictest271833-21/usr/sbin/munin12:40:010
66328994435,5cyclictest189165-21timerwakeupswit10:56:091
66328994434,6cyclictest3459337-21cups-browsed09:30:551
66328994433,7cyclictest1055-21mta-sts-daemon11:11:431
66327994436,5cyclictest421-21dbus-daemon11:40:010
66327994428,13cyclictest155145-21munin-plugin-st10:15:010
66329994335,5cyclictest421-21dbus-daemon07:55:002
66329994331,7cyclictest199372-21cron11:10:002
66328994336,3cyclictest0-21swapper/110:05:591
6632799434,29cyclictest128094-21ssh09:41:020
6632799434,29cyclictest128094-21ssh09:41:020
66329994235,3cyclictest0-21swapper/209:40:012
66329994234,5cyclictest152144-21kernelversion10:10:532
66328994233,4cyclictest0-21swapper/111:15:591
66328994231,6cyclictest0-21swapper/112:36:001
66327994232,7cyclictest0-21swapper/011:45:010
66329994136,2cyclictest0-21swapper/211:20:012
66329994133,5cyclictest223585-21munin-run11:40:002
66328994128,9cyclictest102310-21diskmemload11:20:591
66328994118,7cyclictest0-21swapper/108:16:031
66329994034,3cyclictest0-21swapper/210:45:012
66329994029,7cyclictest0-21swapper/209:15:472
6632899408,5cyclictest102310-21diskmemload10:36:051
6632899404,28cyclictest140250-21ssh09:56:011
6632899403,27cyclictest220929-21ssh11:35:581
66328994032,5cyclictest183437-21latency_hist10:50:031
66328994031,6cyclictest600-21gmain10:20:001
66327994033,4cyclictest159202-21munin-run10:20:000
66329993931,5cyclictest99346-21latency_hist09:00:022
66329993931,5cyclictest227630-21systemd-run11:45:012
66329993929,6cyclictest0-21swapper/212:15:012
66328993912,8cyclictest171rcu_preempt08:36:021
66327993932,4cyclictest235631-21cron11:55:010
66329993832,3cyclictest0-21swapper/211:00:012
66329993831,3cyclictest0-21swapper/210:20:002
66329993828,7cyclictest255776-21/usr/sbin/munin12:20:022
6632899384,27cyclictest136159-21ssh09:51:001
66328993830,5cyclictest255762-21cron12:20:011
66328993829,6cyclictest212911-21ntpq11:25:591
66328993829,6cyclictest212911-21ntpq11:25:591
66328993829,5cyclictest248629-21fschecks_count12:10:481
66327993832,3cyclictest1056-21mta-sts-daemon11:56:050
66327993830,5cyclictest421-21dbus-daemon09:25:010
66327993830,5cyclictest421-21dbus-daemon09:10:020
66327993824,12cyclictest167210-21munin-plugin-st10:30:010
66329993730,3cyclictest0-21swapper/210:05:552
66329993729,4cyclictest0-21swapper/210:00:012
66329993728,6cyclictest195367-21basename11:05:002
66329993728,6cyclictest163945-21aten_r2power_fr10:25:482
66329993727,6cyclictest0-21swapper/210:55:032
6632999372,29cyclictest233184-21ssh11:51:082
66328993730,4cyclictest126573-21cron09:40:011
66328993729,5cyclictest67831-21systemd-run07:15:011
6632899372,29cyclictest144269-21ssh10:00:581
66328993714,6cyclictest171rcu_preempt08:40:571
66327993731,4cyclictest97713-21unixbench_singl08:51:080
66327993729,5cyclictest3459337-21cups-browsed11:34:200
66327993728,6cyclictest3459337-21cups-browsed10:03:240
6632999363,26cyclictest180633-21ssh10:46:002
66329993630,3cyclictest0-21swapper/212:30:162
66329993630,3cyclictest0-21swapper/212:30:162
66329993628,4cyclictest0-21swapper/211:11:112
66328993629,3cyclictest0-21swapper/112:05:031
66328993629,3cyclictest0-21swapper/109:21:011
66328993628,5cyclictest257601-21timerandwakeup12:21:081
66328993628,5cyclictest171230-21cron10:35:011
66328993627,6cyclictest75344-21systemd-run07:40:011
6632999354,24cyclictest103668-21ssh09:10:592
6632999354,24cyclictest103668-21ssh09:10:592
66329993528,4cyclictest259827-21munin-plugin-st12:25:012
66329993526,6cyclictest85818-21systemd-run08:15:012
66329993524,8cyclictest78322-21sh07:50:012
66328993529,3cyclictest0-21swapper/107:10:011
66328993527,5cyclictest130796-21latency_hist09:45:031
66328993527,5cyclictest130796-21latency_hist09:45:031
66328993525,4cyclictest0-21swapper/110:20:581
66327993531,2cyclictest0-21swapper/011:20:010
66327993529,2cyclictest0-21swapper/011:11:160
66327993528,4cyclictest231660-21ssh11:50:010
66327993527,5cyclictest421-21dbus-daemon09:40:010
66327993527,5cyclictest132678-21unixbench_singl09:46:100
66327993527,5cyclictest132678-21unixbench_singl09:46:100
66327993525,7cyclictest1-21systemd09:55:490
6632799352,25cyclictest659-21snmpd08:06:000
6632999343,23cyclictest208919-21ssh11:21:002
66329993426,4cyclictest79807-21kworker/u8:2-xprtiod10:03:022
66329993425,6cyclictest172168-21uniq10:35:502
66328993427,4cyclictest0-21swapper/111:10:011
66328993427,3cyclictest0-21swapper/111:30:561
66328993418,5cyclictest420-21avahi-daemon12:25:311
6632899341,26cyclictest192764-21ssh11:00:591
66327993428,4cyclictest1051-21mta-sts-daemon10:55:560
66327993427,5cyclictest75338-21systemd-run07:40:010
66327993427,5cyclictest66326-21cyclictest09:30:550
66327993426,5cyclictest159858-21/usr/sbin/munin10:20:520
6632799341,27cyclictest168585-21sh10:30:590
66329993327,3cyclictest0-21swapper/212:36:122
66329993325,4cyclictest0-21swapper/211:30:012
66329993325,4cyclictest0-21swapper/211:30:012
66329993324,6cyclictest163232-21systemd-run10:25:012
66329993322,8cyclictest261174-21collect212:26:002
6632999332,24cyclictest127951-21sh09:40:572
6632999332,24cyclictest127951-21sh09:40:572
6632899336,8cyclictest171rcu_preempt08:45:571
6632899336,8cyclictest171rcu_preempt08:45:571
66328993325,5cyclictest85818-21systemd-run08:15:011
66328993325,5cyclictest110369-21awk09:20:001
66328993323,8cyclictest177130-21timerwakeupswit10:41:131
66328993322,8cyclictest79148-21ntp_kernel_err07:50:561
6632899332,24cyclictest547-21gmain08:25:591
6632799333,25cyclictest180633-21ssh10:46:000
66327993328,2cyclictest0-21swapper/007:35:010
66327993327,3cyclictest88812-21cron08:25:010
66327993326,5cyclictest3459334-21cupsd11:01:480
66327993326,4cyclictest90346-21latency_hist08:30:020
66327993326,4cyclictest211530-21systemd-run11:25:010
66327993325,5cyclictest3459337-21cups-browsed10:35:530
66329993227,2cyclictest0-21swapper/208:40:012
66329993226,3cyclictest0-21swapper/211:50:012
66329993226,3cyclictest0-21swapper/209:46:082
66329993226,3cyclictest0-21swapper/209:46:082
66329993225,5cyclictest0-21swapper/207:15:022
66329993225,4cyclictest1056-21mta-sts-daemon07:21:032
66329993225,4cyclictest1054-21mta-sts-daemon09:06:002
66329993223,5cyclictest171236-21munin-plugin-st10:35:012
6632899323,21cyclictest695-21cron11:51:011
66328993225,4cyclictest236366-21aten_r2power_fr11:55:491
66328993223,7cyclictest231652-21sh11:50:011
66328993222,7cyclictest245175-21collect212:06:031
6632899322,23cyclictest224981-21ssh11:40:581
66328993221,8cyclictest131983-21sed09:45:561
66328993221,8cyclictest131983-21sed09:45:561
66327993227,3cyclictest87237-21unixbench_singl08:16:090
66327993226,3cyclictest0-21swapper/009:26:100
66327993225,5cyclictest179254-21cron10:45:010
66327993225,4cyclictest77350-21fschecks_time07:45:520
66327993225,3cyclictest0-21swapper/012:14:530
6632799321,9cyclictest0-21swapper/009:15:550
66327993218,13cyclictest66326-21cyclictest11:07:460
66327993217,11cyclictest3459337-21cups-browsed10:50:140
66329993125,3cyclictest0-21swapper/211:31:112
66329993124,4cyclictest1056-21mta-sts-daemon12:01:022
66329993124,3cyclictest0-21swapper/209:55:002
66329993122,3cyclictest0-21swapper/208:41:002
66328993125,3cyclictest0-21swapper/112:35:001
66328993125,3cyclictest0-21swapper/112:35:001
66328993123,5cyclictest152415-21ssh10:10:581
66327993123,3cyclictest0-21swapper/012:25:010
6632999303,22cyclictest244712-21/usr/sbin/munin12:05:522
66329993025,2cyclictest0-21swapper/208:00:012
66329993024,3cyclictest0-21swapper/208:25:012
66329993022,4cyclictest91684-21unixbench_multi08:31:122
66328993022,5cyclictest102329-21ssh09:10:011
66328993020,8cyclictest73812-21/usr/sbin/munin07:35:011
66327993025,2cyclictest0-21swapper/007:20:010
66327993023,3cyclictest0-21swapper/012:00:560
66327993021,6cyclictest420-21avahi-daemon09:13:290
66327993021,6cyclictest420-21avahi-daemon09:13:290
66327993020,7cyclictest252995-21grep12:15:540
66327993015,11cyclictest151ktimers/008:36:020
6632999293,18cyclictest111761-21ssh09:20:582
66329992915,8cyclictest100238-21cron09:01:012
6632899299,13cyclictest3459337-21cups-browsed07:56:011
66328992920,7cyclictest81852-21gltestperf08:00:531
66328992919,5cyclictest0-21swapper/107:46:011
66327992923,3cyclictest0-21swapper/009:52:120
6632799292,21cyclictest148318-21ssh10:05:560
6632999284,19cyclictest119914-21ssh09:31:002
66329992818,5cyclictest0-21swapper/209:25:592
66327992821,5cyclictest66855-21gltestperf07:10:540
66327992821,3cyclictest0-21swapper/012:05:330
66327992821,3cyclictest0-21swapper/011:30:020
66327992821,3cyclictest0-21swapper/011:30:020
66329992721,3cyclictest0-21swapper/208:51:052
66329992719,5cyclictest82819-21sh08:05:012
66329992719,5cyclictest1055-21mta-sts-daemon08:15:192
6632899274,5cyclictest171rcu_preempt08:50:571
66328992720,4cyclictest88815-21sh08:25:011
66328992718,6cyclictest439-21systemd-logind08:35:011
66328992718,5cyclictest68179-21aten_r2power_po07:15:481
66328992716,8cyclictest98025-21/usr/sbin/munin08:55:541
66329992618,5cyclictest3459337-21cups-browsed08:07:472
6632899264,7cyclictest171rcu_preempt07:26:041
66327992618,6cyclictest516-21in:imuxsock08:40:410
66327992617,7cyclictest1051-21mta-sts-daemon07:30:010
6632999256,11cyclictest428-21mta-sts-daemon08:25:582
66329992517,5cyclictest75296-21sh07:40:002
66329992516,6cyclictest72328-21systemd-run07:30:012
66329992513,9cyclictest753-21ntpd07:15:582
66328992518,5cyclictest69665-21aten_r2power_fr07:20:481
66327992520,3cyclictest1056-21mta-sts-daemon07:55:490
66327992520,2cyclictest0-21swapper/009:05:000
66328992419,2cyclictest0-21swapper/108:10:011
66328992418,4cyclictest1056-21mta-sts-daemon09:01:021
66328992418,3cyclictest0-21swapper/107:45:011
66328992418,3cyclictest0-21swapper/107:45:011
6632799243,18cyclictest69633-21tail07:20:480
66329992315,5cyclictest72851-21fschecks_time07:30:502
66327992318,3cyclictest1056-21mta-sts-daemon08:00:500
66327992317,4cyclictest3459337-21cups-browsed08:56:130
6632799230,3cyclictest171rcu_preempt07:41:000
6632799230,3cyclictest171rcu_preempt07:41:000
66329992217,2cyclictest0-21swapper/207:10:012
6632799221,7cyclictest79370-21sendmail_mailqu07:51:040
66327992217,3cyclictest1051-21mta-sts-daemon08:45:520
66327992217,3cyclictest1051-21mta-sts-daemon08:45:520
66329992113,3cyclictest0-21swapper/208:45:572
66329992113,3cyclictest0-21swapper/208:45:572
66327992117,2cyclictest0-21swapper/008:10:550
6632799202,6cyclictest1-21systemd07:10:010
6633099172,8cyclictest0-21swapper/309:45:573
6633099172,8cyclictest0-21swapper/309:45:573
6633099152,8cyclictest0-21swapper/309:40:563
6633099152,8cyclictest0-21swapper/309:40:563
6633099151,8cyclictest0-21swapper/309:11:003
6633099151,8cyclictest0-21swapper/309:11:003
6633099151,8cyclictest0-21swapper/307:56:003
6633099151,7cyclictest0-21swapper/310:41:013
6633099151,7cyclictest0-21swapper/307:41:013
6633099151,7cyclictest0-21swapper/307:41:013
6633099143,4cyclictest0-21swapper/307:16:003
6633099142,7cyclictest0-21swapper/312:00:563
6633099142,7cyclictest0-21swapper/310:20:483
6633099142,5cyclictest0-21swapper/308:01:023
6633099141,7cyclictest0-21swapper/312:06:043
6633099141,7cyclictest0-21swapper/311:10:483
6633099141,7cyclictest0-21swapper/310:06:003
6633099141,6cyclictest0-21swapper/308:15:563
6633099141,6cyclictest0-21swapper/307:50:583
6633099133,6cyclictest0-21swapper/310:20:003
6633099133,5cyclictest0-21swapper/311:25:523
6633099133,5cyclictest0-21swapper/311:25:523
6633099132,9cyclictest0-21swapper/309:25:473
6633099132,6cyclictest0-21swapper/311:50:553
6633099132,6cyclictest0-21swapper/311:40:513
6633099132,6cyclictest0-21swapper/309:55:413
6633099131,7cyclictest0-21swapper/312:36:073
6633099131,7cyclictest0-21swapper/312:30:573
6633099131,7cyclictest0-21swapper/312:30:573
6633099131,7cyclictest0-21swapper/311:45:503
6633099131,6cyclictest0-21swapper/312:10:443
6633099131,6cyclictest0-21swapper/311:20:583
6633099131,6cyclictest0-21swapper/309:25:023
6633099131,6cyclictest0-21swapper/308:26:003
6633099131,5cyclictest0-21swapper/308:50:553
6633099130,7cyclictest0-21swapper/308:21:003
6633099123,5cyclictest0-21swapper/309:15:463
6633099123,4cyclictest0-21swapper/307:36:163
6633099122,6cyclictest0-21swapper/312:15:563
6633099122,6cyclictest0-21swapper/311:30:503
6633099122,6cyclictest0-21swapper/310:55:513
6633099122,6cyclictest0-21swapper/310:10:573
6633099122,6cyclictest0-21swapper/310:01:013
6633099122,6cyclictest0-21swapper/309:50:473
6633099121,7cyclictest0-21swapper/311:56:053
6633099121,7cyclictest0-21swapper/311:05:553
6633099121,7cyclictest0-21swapper/310:36:013
6633099121,7cyclictest0-21swapper/309:35:553
6633099121,7cyclictest0-21swapper/309:30:533
6633099121,6cyclictest0-21swapper/312:20:563
6633099121,6cyclictest0-21swapper/310:30:513
6633099121,6cyclictest0-21swapper/308:45:553
6633099121,6cyclictest0-21swapper/308:45:553
6633099121,6cyclictest0-21swapper/308:35:593
6633099121,6cyclictest0-21swapper/307:11:043
6633099121,5cyclictest0-21swapper/309:00:583
6633099121,5cyclictest0-21swapper/307:20:563
6633099111,6cyclictest0-21swapper/312:25:473
6633099111,6cyclictest0-21swapper/311:35:553
6633099111,6cyclictest0-21swapper/310:45:583
6633099111,6cyclictest0-21swapper/310:25:483
6633099111,6cyclictest0-21swapper/309:10:033
6633099111,5cyclictest0-21swapper/308:10:563
6633099111,5cyclictest0-21swapper/308:05:573
6633099104,4cyclictest0-21swapper/311:00:413
6633099102,6cyclictest0-21swapper/310:50:593
6633099102,5cyclictest0-21swapper/307:30:023
6633099102,4cyclictest0-21swapper/308:55:573
6633099101,5cyclictest0-21swapper/311:16:033
6633099101,5cyclictest0-21swapper/307:45:513
663309991,5cyclictest0-21swapper/308:41:003
663309983,4cyclictest0-21swapper/307:30:553
663309980,5cyclictest0-21swapper/308:30:583
663309971,3cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional