You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 19:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Sun May 03, 2026 12:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1113399998268,12cyclictest150-21pr/ttyS211:35:020
111341199801,49cyclictest1136652-21ssh09:45:353
111341199801,49cyclictest1136652-21ssh09:45:343
111341199792,47cyclictest1176430-21munin-run11:10:003
111341199791,51cyclictest1190468-21ssh11:39:333
1113411997819,30cyclictest1116793-21cat08:15:013
111341199752,47cyclictest1211336-21ssh12:23:313
111341199751,68cyclictest48-21kcompactd007:45:193
111341199742,46cyclictest1170832-21ssh10:57:583
111341199742,44cyclictest1212186-21ssh12:25:133
1113399997468,2cyclictest0-21swapper/009:40:020
1113411997134,34cyclictest37-21ksoftirqd/307:45:023
111341199711,41cyclictest1153457-21ssh10:20:553
111341199711,41cyclictest1130309-21ssh09:32:203
1113399997156,10cyclictest0-21swapper/008:35:030
111341199705,3cyclictest131rcu_preempt12:30:023
111341199692,39cyclictest1119574-21diskmemload10:45:373
111341199692,39cyclictest1119574-21diskmemload10:45:363
1113399996825,15cyclictest275-21dbus-daemon12:25:010
111341199671,5cyclictest131rcu_preempt10:30:013
111341199651,60cyclictest1183446-21ssh11:24:453
1113411996428,32cyclictest1199219-21sh11:57:473
111341199642,58cyclictest3467602-21systemd-journal12:09:533
111341199641,6cyclictest1191676-21ssh11:41:453
111341199641,59cyclictest1184565-21ssh11:26:473
111341199641,59cyclictest1180673-21ssh11:18:503
111341199641,58cyclictest48-21kcompactd009:15:133
111341199641,58cyclictest48-21kcompactd009:15:123
1113404996418,31cyclictest23-21ksoftirqd/111:55:021
1113399996422,36cyclictest12-21ksoftirqd/010:20:010
1113411996330,28cyclictest1127047-21sh09:25:253
111341199632,57cyclictest1176870-21ssh11:10:283
111341199632,56cyclictest1197801-21ssh11:55:013
1113411996317,42cyclictest37-21ksoftirqd/312:02:583
111341199631,58cyclictest1151593-21ssh10:17:033
111341199631,58cyclictest1143967-21ssh10:00:533
111341199631,58cyclictest1114111-21sshd07:24:103
111341199631,57cyclictest3467602-21systemd-journal11:01:503
111341199631,57cyclictest3467602-21systemd-journal11:01:493
111341199631,57cyclictest1163550-21ssh10:42:293
1113399996348,11cyclictest12-21ksoftirqd/011:53:090
111341199624,39cyclictest1141144-21ssh09:55:013
111341199622,56cyclictest1148975-21ssh10:11:293
111341199622,56cyclictest1142989-21ssh09:59:133
111341199622,56cyclictest1142989-21ssh09:59:123
111341199622,56cyclictest1113894-21/usr/sbin/munin07:20:013
1113411996222,35cyclictest1132688-21sh09:37:203
111341199621,41cyclictest1194886-21ssh11:48:433
1113399996232,13cyclictest1212237-21ssh12:25:190
111341199611,8cyclictest1208134-21ssh12:16:473
111341199611,56cyclictest1218679-21ssh12:39:393
111341199611,56cyclictest1205163-21ssh12:10:123
1113399996111,30cyclictest131rcu_preempt10:53:340
1113399996111,30cyclictest131rcu_preempt10:53:330
111341199602,54cyclictest1146093-21ssh10:05:253
111341199602,54cyclictest1146093-21ssh10:05:243
111341199601,55cyclictest1158446-21ssh10:31:373
111341199601,55cyclictest1134434-21ssh09:40:503
111341199601,55cyclictest1125400-21ssh09:22:053
1113408996048,8cyclictest1150514-21kworker/2:210:34:012
1113399996028,16cyclictest1142776-21ssh09:58:470
1113399996028,16cyclictest1142776-21ssh09:58:460
111341199592,53cyclictest1113605-21munin-run07:15:023
111341199592,53cyclictest1113605-21munin-run07:15:013
111341199592,52cyclictest1117501-21munin-run08:30:003
111341199592,51cyclictest1114635-21munin-run07:35:003
111341199591,54cyclictest1167265-21ssh10:50:103
111341199591,54cyclictest1167265-21ssh10:50:093
111341199591,54cyclictest1121272-21ssh09:13:363
1113399995939,17cyclictest12-21ksoftirqd/009:20:100
111341199582,52cyclictest1118799-21munin-run08:55:003
111341199581,53cyclictest1187361-21ssh11:32:443
1113411995815,38cyclictest275-21dbus-daemon08:10:023
1113411995815,38cyclictest275-21dbus-daemon08:10:013
111339999589,40cyclictest12-21ksoftirqd/011:18:490
1113399995837,14cyclictest3467602-21systemd-journal07:15:000
1113399995837,14cyclictest3467602-21systemd-journal07:15:000
1113399995817,37cyclictest12-21ksoftirqd/010:05:070
1113399995817,37cyclictest12-21ksoftirqd/010:05:060
1113399995812,38cyclictest12-21ksoftirqd/011:04:560
1113399995812,38cyclictest12-21ksoftirqd/011:04:560
1113399995812,37cyclictest12-21ksoftirqd/012:33:490
111341199572,51cyclictest1116944-21sshd08:18:103
111341199571,51cyclictest1160214-21sshd10:35:093
1113399995732,9cyclictest1201264-21ssh12:02:070
1113399995730,12cyclictest1175041-21ssh11:06:430
1113399995727,26cyclictest12-21ksoftirqd/010:59:030
1113399995725,14cyclictest1204870-21ssh12:10:020
111341199563,49cyclictest1116215-21munin-run08:05:013
111341199561,51cyclictest3467715-21lldpd08:49:413
111341199561,50cyclictest1119080-21munin-run09:00:013
1113408995647,5cyclictest1117763-21kworker/2:109:24:142
1113408995638,13cyclictest275-21dbus-daemon07:50:002
1113404995649,4cyclictest0-21swapper/109:15:021
111339999569,5cyclictest131rcu_preempt10:35:090
1113399995643,10cyclictest12-21ksoftirqd/011:30:010
1113399995626,14cyclictest1122541-21ssh09:16:040
1113399995626,14cyclictest1122541-21ssh09:16:040
1113399995611,40cyclictest12-21ksoftirqd/009:09:590
111341199552,48cyclictest364-21snmpd07:35:133
1113411995522,29cyclictest1115964-21systemd-run08:00:013
111341199551,49cyclictest3467602-21systemd-journal07:55:003
111341199551,49cyclictest3467602-21systemd-journal07:55:003
111341199551,49cyclictest1118268-21sshd08:44:463
1113399995527,13cyclictest1177878-21ssh11:12:450
111339999552,18cyclictest1218868-21munin-run12:40:010
1113399995512,8cyclictest12-21ksoftirqd/010:26:570
111341199543,45cyclictest1119600-21munin-run09:10:003
1113399995430,7cyclictest16850irq/134-800000009:30:000
1113399995428,11cyclictest1154575-21ssh10:23:270
1113399995426,13cyclictest1188592-21ssh11:35:100
1113399995420,11cyclictest12-21ksoftirqd/012:20:010
1113408995336,8cyclictest1169405-21kworker/2:111:51:562
1113408995330,15cyclictest275-21dbus-daemon07:35:002
1113404995336,13cyclictest1-21systemd08:15:001
1113399995324,13cyclictest1120936-21ssh09:12:540
111339999531,37cyclictest1159532-21ssh10:34:110
111341199521,4cyclictest1117718-21sshd08:31:103
111341199521,44cyclictest1119340-21wc09:05:023
1113408995243,6cyclictest1117763-21kworker/2:109:31:562
1113408995241,6cyclictest1169405-21kworker/2:112:38:062
1113408995235,14cyclictest3467602-21systemd-journal12:34:112
111339999523,7cyclictest3467602-21systemd-journal11:40:110
1113399995225,12cyclictest1137087-21ssh09:46:290
1113399995225,12cyclictest1137087-21ssh09:46:290
111339999522,44cyclictest1207166-21cron12:15:010
1113399995220,29cyclictest12-21ksoftirqd/010:04:560
111341199511,46cyclictest364-21snmpd08:37:583
1113408995143,5cyclictest1150514-21kworker/2:210:39:132
1113408995142,6cyclictest1169405-21kworker/2:111:25:562
1113408995142,6cyclictest1118296-21kworker/2:009:42:012
1113408995141,6cyclictest1115445-21kworker/2:308:34:592
1113399995128,8cyclictest1139407-21ssh09:51:230
1113399995124,12cyclictest1129744-21ssh09:31:090
1113408995042,5cyclictest1145714-21kworker/2:311:01:042
1113408995042,5cyclictest1145714-21kworker/2:311:01:042
1113408995042,5cyclictest1145714-21kworker/2:310:48:082
1113408995042,5cyclictest1145714-21kworker/2:310:48:072
1113408995041,6cyclictest1118296-21kworker/2:009:56:172
1113408995041,6cyclictest1118296-21kworker/2:009:56:162
1113408995040,6cyclictest1169405-21kworker/2:111:45:012
1113408995039,7cyclictest1169405-21kworker/2:111:47:462
1113408995034,11cyclictest1-21systemd10:15:022
111340499501,4cyclictest131rcu_preempt10:55:001
111340499501,4cyclictest131rcu_preempt10:54:591
1113399995028,8cyclictest1132951-21ssh09:38:100
111339999501,7cyclictest275-21dbus-daemon07:55:010
111339999501,7cyclictest275-21dbus-daemon07:55:000
111341199491,44cyclictest364-21snmpd07:26:193
1113408994941,5cyclictest1150514-21kworker/2:210:40:272
1113408994941,5cyclictest1150514-21kworker/2:210:29:502
1113408994941,5cyclictest1118296-21kworker/2:009:55:002
1113408994940,5cyclictest1169405-21kworker/2:111:17:212
1113408994940,5cyclictest1150514-21kworker/2:210:17:312
1113408994912,6cyclictest131rcu_preempt12:14:482
1113399994939,7cyclictest16950irq/224-800000012:00:010
111341199482,42cyclictest364-21snmpd08:22:553
111340899485,40cyclictest291rcuc/209:00:022
111340899484,27cyclictest1114862-21kworker/u8:107:45:032
1113408994840,5cyclictest1169405-21kworker/2:111:12:492
1113408994840,5cyclictest1118296-21kworker/2:010:08:032
1113408994840,5cyclictest1118296-21kworker/2:010:08:022
1113408994840,5cyclictest1118296-21kworker/2:010:03:212
1113408994839,6cyclictest1197815-21kworker/2:012:09:522
1113408994831,13cyclictest275-21dbus-daemon10:55:002
1113408994831,13cyclictest275-21dbus-daemon10:55:002
111340899482,18cyclictest1208887-21ssh12:18:312
111340499486,21cyclictest0-21swapper/109:32:521
1113399994812,31cyclictest16850irq/134-800000010:45:030
1113408994739,5cyclictest1169405-21kworker/2:111:07:562
1113408994739,5cyclictest1118296-21kworker/2:009:47:082
1113408994739,5cyclictest1118296-21kworker/2:009:47:072
1113408994738,6cyclictest1169405-21kworker/2:112:02:302
1113408994737,6cyclictest1117763-21kworker/2:109:15:512
1113408994737,6cyclictest1117763-21kworker/2:109:15:512
1113408994731,11cyclictest309-21in:imuxsock09:05:022
1113404994734,10cyclictest1-21systemd10:10:021
1113404994734,10cyclictest1-21systemd10:10:011
1113404994733,11cyclictest1199312-21sshd11:58:101
1113404994733,11cyclictest1140975-21pluginstate09:55:011
1113399994725,16cyclictest12-21ksoftirqd/008:20:000
111339999471,41cyclictest364-21snmpd11:48:300
111340899468,5cyclictest131rcu_preempt10:20:062
1113408994639,5cyclictest1145714-21kworker/2:310:55:542
1113408994638,5cyclictest1145714-21kworker/2:311:31:222
1113408994636,7cyclictest1197815-21kworker/2:012:29:592
1113404994632,11cyclictest23-21ksoftirqd/112:34:171
1113404994629,13cyclictest1-21systemd11:39:591
1113404994629,13cyclictest1-21systemd11:10:001
1113404994629,12cyclictest275-21dbus-daemon08:35:011
1113399994629,14cyclictest3467602-21systemd-journal10:45:020
1113399994629,14cyclictest3467602-21systemd-journal10:45:020
1113399994628,13cyclictest1117474-21sshd08:27:100
1113399994627,15cyclictest1119061-21tr09:00:010
1113399994621,21cyclictest12-21ksoftirqd/008:05:010
1113408994538,5cyclictest1117763-21kworker/2:109:27:462
1113408994537,5cyclictest1118296-21kworker/2:009:40:002
1113408994537,5cyclictest1112726-21kworker/2:107:11:102
1113408994537,5cyclictest1112726-21kworker/2:107:11:102
1113408994537,5cyclictest1104634-21kworker/2:207:18:102
1113408994536,5cyclictest1169405-21kworker/2:112:22:552
111340499454,34cyclictest1119578-21munin-run09:10:011
111340499451,24cyclictest0-21swapper/112:24:271
1113404994510,6cyclictest131rcu_preempt10:30:131
111339999458,33cyclictest12-21ksoftirqd/008:40:100
111339999455,36cyclictest0-21swapper/008:39:590
1113399994535,6cyclictest16950irq/224-800000008:00:000
1113399994528,13cyclictest1148515-21ssh10:10:310
111339999452,7cyclictest1114886-21sh07:40:000
1113399994520,12cyclictest1181680-21ssh11:20:340
111341199442,37cyclictest1113490-21date07:10:013
1113408994437,4cyclictest1117763-21kworker/2:109:13:342
1113408994436,5cyclictest1145714-21kworker/2:311:56:292
1113399994437,4cyclictest16950irq/224-800000008:50:010
111339999441,3cyclictest131rcu_preempt08:25:000
111340899438,6cyclictest131rcu_preempt11:35:192
1113408994335,5cyclictest1117763-21kworker/2:108:54:592
111340899432,3cyclictest131rcu_preempt11:21:082
1113404994332,8cyclictest1117480-21sshd08:29:091
1113404994330,10cyclictest23-21ksoftirqd/109:20:011
1113404994330,10cyclictest23-21ksoftirqd/109:20:001
1113404994325,12cyclictest275-21dbus-daemon08:40:011
111340499432,12cyclictest1124564-21ssh09:20:101
111340499431,4cyclictest131rcu_preempt12:11:361
111340499431,4cyclictest131rcu_preempt11:14:321
1113408994227,11cyclictest1-21systemd07:30:022
111340499425,6cyclictest131rcu_preempt11:35:011
111340499423,6cyclictest131rcu_preempt11:20:021
1113404994229,9cyclictest23-21ksoftirqd/112:40:001
1113404994225,10cyclictest3467602-21systemd-journal08:20:001
111340499421,7cyclictest1143300-21cron10:00:001
111340499421,7cyclictest1143300-21cron10:00:001
111340499421,13cyclictest1185385-21ssh11:28:411
111340899416,6cyclictest131rcu_preempt07:37:102
111340899415,7cyclictest30-21ksoftirqd/208:50:022
1113408994129,7cyclictest1117992-21cron08:39:592
111340499414,6cyclictest131rcu_preempt11:42:091
111340499414,3cyclictest131rcu_preempt11:00:011
111340499414,3cyclictest131rcu_preempt10:48:551
111340499414,3cyclictest131rcu_preempt10:48:541
111340499414,3cyclictest131rcu_preempt09:44:181
111340499413,3cyclictest131rcu_preempt09:40:011
1113404994130,8cyclictest23-21ksoftirqd/110:20:451
1113404994129,9cyclictest23-21ksoftirqd/110:39:281
1113404994129,9cyclictest23-21ksoftirqd/107:45:021
1113404994125,12cyclictest1-21systemd12:20:021
111340499411,6cyclictest0-21swapper/110:10:311
111340499411,4cyclictest131rcu_preempt07:59:591
1113404994110,10cyclictest0-21swapper/110:28:231
1113399994126,11cyclictest12-21ksoftirqd/007:50:000
1113399994124,12cyclictest275-21dbus-daemon07:25:000
1113399994120,14cyclictest1-21systemd07:30:010
1113408994024,12cyclictest30-21ksoftirqd/208:10:032
1113408994024,12cyclictest30-21ksoftirqd/208:10:022
111340499403,3cyclictest131rcu_preempt08:56:101
1113404994029,9cyclictest23-21ksoftirqd/109:49:541
1113404994029,9cyclictest23-21ksoftirqd/109:49:541
1113404994028,10cyclictest23-21ksoftirqd/111:00:291
1113404994028,10cyclictest23-21ksoftirqd/111:00:291
1113404994027,10cyclictest23-21ksoftirqd/108:25:011
1113404994026,11cyclictest23-21ksoftirqd/111:24:361
1113404994026,10cyclictest23-21ksoftirqd/109:27:371
111340499402,3cyclictest131rcu_preempt10:16:351
1113404994021,14cyclictest275-21dbus-daemon10:05:001
111340499401,8cyclictest0-21swapper/110:44:591
111340499401,4cyclictest131rcu_preempt12:01:341
111339999402,5cyclictest1115166-21munin-run07:45:000
1113408993931,6cyclictest30-21ksoftirqd/207:25:002
1113404993925,11cyclictest23-21ksoftirqd/108:05:011
1113404993924,11cyclictest1114306-21wc07:25:031
1113399993929,8cyclictest12-21ksoftirqd/007:20:010
1113399993929,7cyclictest12-21ksoftirqd/009:05:010
111340499384,3cyclictest131rcu_preempt07:17:101
1113404993827,9cyclictest23-21ksoftirqd/107:55:011
1113404993827,9cyclictest23-21ksoftirqd/107:55:001
1113404993825,10cyclictest275-21dbus-daemon07:30:011
111340499382,3cyclictest131rcu_preempt12:27:101
111340499381,4cyclictest131rcu_preempt08:45:011
1113399993810,11cyclictest12-21ksoftirqd/008:10:010
1113399993810,11cyclictest12-21ksoftirqd/008:10:000
111340899372,8cyclictest0-21swapper/208:03:102
1113404993725,9cyclictest23-21ksoftirqd/107:15:021
1113404993725,9cyclictest23-21ksoftirqd/107:15:021
111340499372,3cyclictest131rcu_preempt12:05:581
111340499372,3cyclictest131rcu_preempt07:50:011
1113404993723,10cyclictest23-21ksoftirqd/107:35:001
111340499371,4cyclictest131rcu_preempt11:45:581
111339999373,30cyclictest0-21swapper/007:30:050
1113399993725,8cyclictest1116674-21sshd08:11:100
111339999371,32cyclictest0-21swapper/008:55:000
111340899365,4cyclictest131rcu_preempt08:15:012
1113408993628,5cyclictest3467562-21sshd07:56:092
1113408993625,8cyclictest1119288-21sshd09:02:102
1113408993621,11cyclictest275-21dbus-daemon08:30:012
111340499363,3cyclictest131rcu_preempt08:55:021
111340499363,3cyclictest131rcu_preempt08:10:021
111340499363,3cyclictest131rcu_preempt08:10:011
1113404993626,8cyclictest23-21ksoftirqd/107:10:011
111340499361,16cyclictest1114391-21kworker/u8:307:40:021
1113408993527,5cyclictest1113334-21latency_hist07:10:012
111340899352,6cyclictest364-21snmpd08:15:322
1113404993525,7cyclictest23-21ksoftirqd/109:05:011
1113404993525,7cyclictest1118742-21cut08:50:021
1113408993426,5cyclictest1117763-21kworker/2:108:43:502
1113399993326,5cyclictest19350irq/34-mmc107:10:020
1113408993225,4cyclictest1115667-21latency_hist07:55:032
1113408993225,4cyclictest1115667-21latency_hist07:55:022
1113408993223,5cyclictest309-21in:imuxsock08:25:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional