You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 13:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed May 13, 2026 12:43:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
935205996357,3cyclictest0-21swapper/109:50:511
935204995547,4cyclictest0-21swapper/007:30:580
935204995538,10cyclictest12350irq/32-eth010:35:590
93520599522,40cyclictest1049099-21ssh10:45:521
935204995237,5cyclictest1133418-21ntp_kernel_pll_12:30:530
935204995038,9cyclictest1085023-21latency11:30:480
935204995038,9cyclictest1085023-21latency11:30:480
935204994942,5cyclictest0-21swapper/009:16:050
935204994938,7cyclictest151ktimers/011:25:500
935205994839,6cyclictest0-21swapper/112:00:001
935205994837,8cyclictest950228-21systemctl08:00:001
935205994837,8cyclictest950228-21systemctl08:00:001
935205994736,4cyclictest0-21swapper/111:30:531
935205994736,4cyclictest0-21swapper/111:30:531
935204994739,5cyclictest1132135-21latency_hist12:30:020
935206994638,5cyclictest935203-21cyclictest12:10:582
935206994617,9cyclictest1097394-21ntpq11:45:552
935205994639,4cyclictest1059848-21sendmail11:00:011
935205994639,4cyclictest1059848-21sendmail11:00:011
935205994635,8cyclictest1117515-21sshd12:10:581
935205994635,5cyclictest0-21swapper/111:05:501
935205994634,6cyclictest0-21swapper/110:00:521
935205994537,5cyclictest261-21systemd-journal12:30:011
935204994538,4cyclictest1035688-21systemd-run10:30:020
93520699444,31cyclictest1029027-21ssh10:20:542
935206994434,6cyclictest1059864-21sh11:00:022
935206994434,6cyclictest1059864-21sh11:00:022
935205994437,4cyclictest0-21swapper/109:20:561
935205994436,5cyclictest1128094-21systemd-run12:25:011
935205994433,9cyclictest1071872-21/usr/sbin/munin11:15:021
935206994334,4cyclictest0-21swapper/208:50:012
935206994333,6cyclictest96104-21cups-browsed10:00:022
935206994332,6cyclictest0-21swapper/210:30:542
935206994332,6cyclictest0-21swapper/210:30:542
935204994337,4cyclictest1128094-21systemd-run12:25:010
935204994335,5cyclictest1057372-21sendmail_mailqu10:56:020
935204994335,5cyclictest1057372-21sendmail_mailqu10:56:020
935204994322,14cyclictest151ktimers/007:35:570
93520699427,26cyclictest972581-21ssh09:10:532
935206994235,3cyclictest0-21swapper/211:30:012
935206994234,5cyclictest1019570-21systemd-run10:10:012
935206994227,11cyclictest1-21systemd12:20:432
935205994234,5cyclictest989106-21unixbench-2d09:31:061
935205994234,5cyclictest1031671-21ssh10:25:011
93520599423,30cyclictest1109297-21ssh12:00:511
935204994231,7cyclictest1097253-21sh11:45:500
935206994133,5cyclictest941268-21systemd-run07:30:022
93520499414,31cyclictest151ktimers/008:40:530
935204994134,4cyclictest1037573-21grep10:31:030
935204994134,4cyclictest1037573-21grep10:31:030
935204994133,5cyclictest1007417-21systemd-run09:55:010
935204994130,8cyclictest964546-21needreboot08:45:480
935204994130,8cyclictest1-21systemd11:20:010
93520699404,28cyclictest980554-21ssh09:20:572
935206994033,4cyclictest1043701-21cron10:40:002
935206994029,8cyclictest421-21dbus-daemon11:45:012
935206994029,7cyclictest421-21dbus-daemon10:14:202
93520599405,27cyclictest1045056-21ssh10:40:521
935204994029,8cyclictest1089900-21unixbench_singl11:36:050
935204994024,11cyclictest0-21swapper/012:15:590
935204994024,11cyclictest0-21swapper/012:15:590
935206993931,5cyclictest939758-21munin-plugin-st07:25:012
935206993931,3cyclictest0-21swapper/209:40:002
935206993929,7cyclictest420-21avahi-daemon11:21:572
935206993929,6cyclictest1055-21mta-sts-daemon09:50:102
935206993925,10cyclictest977044-21timerandwakeup09:16:072
935206993911,8cyclictest970643-21ntpq09:05:502
935206993911,11cyclictest998042-21ssh09:42:322
935205993931,5cyclictest987224-21cron09:30:011
935205993930,6cyclictest439-21systemd-logind08:45:011
935205993929,6cyclictest1043854-21latency_hist10:40:021
935204993932,4cyclictest954719-21cron08:15:010
935204993930,6cyclictest985008-21timerwakeupswit09:26:030
935206993831,4cyclictest0-21swapper/210:20:002
935206993830,5cyclictest1105840-21grep11:56:042
935206993828,6cyclictest1136220-21latency_hist12:35:022
935206993813,7cyclictest967581-21ntpq08:55:502
935206993810,9cyclictest1073306-21ntp_states11:15:562
93520599385,20cyclictest428-21mta-sts-daemon11:45:541
935205993830,5cyclictest1-21systemd09:20:021
935205993830,5cyclictest1116161-21latency_hist12:10:021
935205993830,4cyclictest0-21swapper/109:10:581
935204993830,5cyclictest950257-21systemd-run08:00:010
935204993830,5cyclictest950257-21systemd-run08:00:010
935204993830,5cyclictest1065808-21head11:06:070
935204993824,10cyclictest0-21swapper/012:05:410
935204993820,14cyclictest0-21swapper/011:01:050
935206993713,5cyclictest1089286-21ntpq11:35:552
935206993711,9cyclictest1121463-21ntp_states12:15:582
935206993711,9cyclictest1121463-21ntp_states12:15:582
93520599378,5cyclictest171rcu_preempt07:45:481
935205993729,5cyclictest938284-21systemd-run07:20:021
935205993728,6cyclictest1019631-21latency_hist10:10:021
935205993728,6cyclictest1008221-21cpuspeed09:55:431
935205993725,8cyclictest0-21swapper/112:35:561
93520599372,26cyclictest1061189-21ssh11:00:541
935204993729,5cyclictest953303-21latency_hist08:10:010
935204993729,5cyclictest1055867-21systemd-run10:55:020
93520699367,19cyclictest1000631-21ssh09:45:512
935206993628,5cyclictest949255-21fschecks_time07:55:442
935206993628,5cyclictest949255-21fschecks_time07:55:442
935206993628,5cyclictest1056-21mta-sts-daemon10:43:282
935206993626,7cyclictest1049756-21grep10:46:042
935206993623,9cyclictest420-21avahi-daemon12:39:362
935206993619,13cyclictest957167-21ntpq08:20:542
935206993611,8cyclictest1064543-21ssh11:05:432
935206993610,8cyclictest947243-21awk07:50:002
935205993626,6cyclictest420-21avahi-daemon09:40:241
935204993628,6cyclictest956231-21munin-run08:20:000
935204993625,8cyclictest0-21swapper/009:20:420
93520499362,29cyclictest988535-21ssh09:30:520
93520699359,11cyclictest953228-21latency_hist08:10:022
93520699358,9cyclictest984069-21diskstats09:25:432
93520699356,10cyclictest1131283-21ssh12:28:282
935206993528,4cyclictest695-21cron11:55:002
935206993526,6cyclictest1061401-21grep11:01:002
935206993522,10cyclictest969133-21ntp_kernel_pll_09:00:522
935206993520,10cyclictest1109428-21ntpq12:00:562
935205993529,3cyclictest0-21swapper/107:50:521
935205993527,5cyclictest96104-21cups-browsed07:20:531
935205993527,4cyclictest0-21swapper/110:15:531
935205993525,6cyclictest421-21dbus-daemon10:31:591
935205993525,6cyclictest421-21dbus-daemon10:31:591
935205993524,7cyclictest1121426-21sh12:15:571
935205993524,7cyclictest1121426-21sh12:15:571
935204993528,4cyclictest1079959-21kworker/u8:9+xprtiod12:00:010
935204993523,8cyclictest151ktimers/011:50:550
93520499352,23cyclictest1000791-21ssh09:45:570
93520499352,10cyclictest971223-21diskmemload10:20:530
93520499351,27cyclictest1093315-21ssh11:40:530
93520699347,18cyclictest428-21mta-sts-daemon10:50:492
935206993428,3cyclictest0-21swapper/207:40:002
935206993426,5cyclictest951740-21munin-plugin-st08:05:002
935206993426,5cyclictest951740-21munin-plugin-st08:05:002
935206993410,7cyclictest936074-21ntpq07:10:572
935205993428,3cyclictest0-21swapper/107:40:001
935204993428,3cyclictest0-21swapper/010:05:590
935204993427,4cyclictest997062-21wc09:41:030
935204993427,3cyclictest0-21swapper/012:00:290
935204993426,5cyclictest96104-21cups-browsed10:47:250
93520699336,8cyclictest960096-21ntpq08:30:502
935206993326,5cyclictest935203-21cyclictest10:30:022
935206993322,8cyclictest963084-21ntp_kernel_err08:40:512
935206993322,8cyclictest937609-21ntp_kernel_pll_07:15:522
935205993326,4cyclictest1051-21mta-sts-daemon09:05:481
935205993326,4cyclictest0-21swapper/111:40:001
935205993325,5cyclictest959767-21fschecks_time08:30:441
935205993325,5cyclictest1020152-21aten_r2power_cu10:10:401
935205993324,6cyclictest1076733-21cpuspeed11:20:431
935205993322,8cyclictest1093252-21ntp_kernel_pll_11:40:521
93520599332,25cyclictest1101287-21sshd11:50:541
935205993319,3cyclictest171rcu_preempt11:15:421
93520499338,20cyclictest624-21lldpd09:05:510
93520499334,21cyclictest972513-21ssh09:10:500
935204993326,3cyclictest0-21swapper/009:40:010
93520499332,26cyclictest1012465-21awk10:00:460
93520699329,7cyclictest989928-21ssh09:32:312
93520699329,7cyclictest1115464-21ssh12:08:552
935206993219,8cyclictest942162-21ntpq07:30:542
935206993216,11cyclictest1087678-21ssh11:34:332
935206993216,11cyclictest1087678-21ssh11:34:332
935205993225,4cyclictest0-21swapper/109:40:011
935205993224,5cyclictest935210-21systemd-run07:10:021
935205993223,6cyclictest0-21swapper/108:55:021
935204993225,4cyclictest1117938-21unixbench_multi12:11:040
935204993224,5cyclictest938372-21latency_hist07:20:030
935204993224,5cyclictest1076960-21kernelversion11:20:470
93520699318,17cyclictest0-21swapper/208:35:492
93520699317,8cyclictest958331-21if_eth008:25:452
935206993117,11cyclictest945168-21ntpq07:40:542
935206993117,11cyclictest945168-21ntpq07:40:542
935206993114,12cyclictest1068196-21ssh11:10:152
93520599317,10cyclictest0-21swapper/107:25:521
935205993124,3cyclictest0-21swapper/112:31:011
93520599312,24cyclictest1081300-21ssh11:25:531
935205993121,6cyclictest1003378-21cut09:50:011
935205993120,8cyclictest420-21avahi-daemon10:26:191
93520499313,9cyclictest171rcu_preempt07:25:500
935204993124,4cyclictest962220-21munin-run08:40:010
935204993123,5cyclictest966735-21systemd-run08:55:010
935204993123,5cyclictest151ktimers/010:10:570
935204993123,4cyclictest0-21swapper/007:45:510
935204993118,9cyclictest12350irq/32-eth011:10:590
935204993114,14cyclictest968215-21sendmail09:00:010
93520699305,8cyclictest966191-21ntpq08:50:552
935206993018,7cyclictest955589-21ntpq08:15:502
935206993018,7cyclictest1012811-21ntpq10:00:532
935206993012,4cyclictest954215-21ntpq08:10:562
935205993025,3cyclictest935203-21cyclictest08:45:531
935205993024,3cyclictest0-21swapper/109:00:011
935205993024,2cyclictest0-21swapper/108:40:011
935205993023,3cyclictest0-21swapper/110:50:481
935205992922,4cyclictest951669-21unixbench_singl08:01:031
935205992922,4cyclictest951669-21unixbench_singl08:01:031
935205992919,8cyclictest957550-21timerwakeupswit08:21:031
93520499295,15cyclictest12350irq/32-eth009:55:510
93520499293,15cyclictest151ktimers/008:00:570
93520499293,15cyclictest151ktimers/008:00:570
935204992917,6cyclictest1044377-21aten_r2power_en10:40:410
935204992912,4cyclictest171rcu_preempt12:39:270
93520499291,22cyclictest1024929-21ssh10:15:520
935205992823,3cyclictest0-21swapper/108:20:001
935205992814,9cyclictest1051-21mta-sts-daemon08:10:551
935204992820,5cyclictest151ktimers/009:00:520
935206992715,9cyclictest948077-21ntp_kernel_err07:50:512
935205992718,6cyclictest548-21gdbus08:26:121
93520699264,7cyclictest935241-21cut07:10:022
93520599262,7cyclictest0-21swapper/108:05:491
935205992619,4cyclictest968606-21aten2.4-expect09:00:421
935205992618,5cyclictest695-21cron07:35:011
935204992620,4cyclictest935653-21cpuspeed07:10:450
935204992618,4cyclictest12450irq/33-eth007:20:520
935204992613,10cyclictest96104-21cups-browsed08:25:490
93520599250,7cyclictest171rcu_preempt07:14:141
935204992518,4cyclictest948717-21cron07:55:000
935204992513,9cyclictest0-21swapper/008:30:440
935205992416,6cyclictest271ktimers/107:40:531
935205992416,6cyclictest271ktimers/107:40:531
935204992216,3cyclictest1055-21mta-sts-daemon08:20:440
935204992114,3cyclictest945457-21threads07:41:020
935204992114,3cyclictest945457-21threads07:41:020
93520799202,11cyclictest0-21swapper/312:15:593
93520799202,11cyclictest0-21swapper/312:15:593
93520499192,14cyclictest935163-21munin-run07:10:010
93520799174,7cyclictest0-21swapper/311:35:553
93520799172,9cyclictest0-21swapper/311:30:593
93520799172,9cyclictest0-21swapper/311:30:593
93520799162,8cyclictest0-21swapper/312:00:563
93520799161,9cyclictest0-21swapper/309:35:583
93520799154,5cyclictest0-21swapper/310:30:543
93520799154,5cyclictest0-21swapper/310:30:543
93520799153,8cyclictest0-21swapper/310:10:503
93520799153,7cyclictest0-21swapper/309:45:583
93520799152,7cyclictest0-21swapper/307:30:583
93520799151,7cyclictest0-21swapper/311:20:563
93520799151,6cyclictest0-21swapper/308:45:523
93520799144,7cyclictest0-21swapper/311:45:513
93520799144,6cyclictest0-21swapper/308:25:513
93520799144,4cyclictest0-21swapper/307:50:583
93520799143,6cyclictest0-21swapper/310:20:503
93520799142,7cyclictest0-21swapper/312:10:543
93520799141,9cyclictest0-21swapper/312:25:503
93520799141,8cyclictest0-21swapper/310:51:003
93520799141,8cyclictest0-21swapper/309:40:573
93520799141,7cyclictest0-21swapper/312:30:493
93520799141,7cyclictest0-21swapper/310:45:543
93520799141,7cyclictest0-21swapper/309:15:543
93520799141,7cyclictest0-21swapper/307:40:513
93520799141,7cyclictest0-21swapper/307:40:513
93520799141,6cyclictest0-21swapper/308:55:553
93520799141,6cyclictest0-21swapper/308:10:503
93520799141,6cyclictest0-21swapper/308:05:563
93520799141,6cyclictest0-21swapper/307:35:533
93520799134,6cyclictest0-21swapper/309:10:503
93520799133,7cyclictest0-21swapper/310:00:463
93520799133,6cyclictest0-21swapper/311:06:073
93520799132,7cyclictest0-21swapper/310:25:573
93520799132,5cyclictest0-21swapper/309:00:513
93520799131,8cyclictest0-21swapper/310:35:553
93520799131,8cyclictest0-21swapper/309:25:513
93520799131,7cyclictest0-21swapper/312:20:513
93520799131,7cyclictest0-21swapper/311:55:583
93520799131,7cyclictest0-21swapper/310:05:423
93520799131,7cyclictest0-21swapper/309:30:513
93520799131,7cyclictest0-21swapper/308:40:503
93520799131,6cyclictest0-21swapper/311:25:453
93520799131,6cyclictest0-21swapper/307:20:533
93520799131,6cyclictest0-21swapper/307:15:563
93520799131,5cyclictest0-21swapper/309:05:473
93520799125,5cyclictest0-21swapper/312:05:573
93520799125,5cyclictest0-21swapper/311:40:533
93520799124,6cyclictest0-21swapper/311:10:563
93520799123,6cyclictest0-21swapper/311:01:003
93520799122,7cyclictest0-21swapper/311:50:533
93520799122,6cyclictest0-21swapper/312:35:403
93520799122,6cyclictest0-21swapper/310:40:413
93520799122,6cyclictest0-21swapper/309:20:523
93520799122,6cyclictest0-21swapper/308:50:563
93520799122,6cyclictest0-21swapper/308:40:013
93520799121,7cyclictest0-21swapper/309:55:513
93520799121,7cyclictest0-21swapper/309:50:533
93520799121,7cyclictest0-21swapper/308:00:543
93520799121,7cyclictest0-21swapper/308:00:543
93520799121,6cyclictest0-21swapper/310:15:493
93520799121,6cyclictest0-21swapper/307:55:493
93520799121,6cyclictest0-21swapper/307:55:493
93520799111,6cyclictest0-21swapper/311:15:443
93520799111,6cyclictest0-21swapper/308:20:573
93520799111,6cyclictest0-21swapper/308:15:523
93520799111,6cyclictest0-21swapper/307:45:563
93520799111,5cyclictest0-21swapper/308:30:503
93520799102,5cyclictest0-21swapper/310:55:433
93520799102,5cyclictest0-21swapper/310:55:433
93520799101,5cyclictest0-21swapper/307:10:553
93520799101,4cyclictest0-21swapper/307:30:033
9352079983,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional