You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 17:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Jun 26, 2026 12:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17535509910197,2cyclictest0-21swapper/010:45:020
1753550999992,3cyclictest0-21swapper/008:35:030
1753550999992,3cyclictest0-21swapper/008:35:020
1753553998133,13cyclictest23-21ksoftirqd/112:05:001
175355099741,41cyclictest131rcu_preempt10:51:080
1753558997340,7cyclictest1755322-21kworker/2:010:55:022
1753553997330,11cyclictest23-21ksoftirqd/110:35:011
175355399711,6cyclictest131rcu_preempt11:40:021
175355399711,6cyclictest131rcu_preempt11:40:021
1753550997127,14cyclictest131rcu_preempt10:29:310
1753558996942,9cyclictest1759273-21kworker/2:209:55:012
1753558996838,8cyclictest1759273-21kworker/2:210:59:372
1753558996742,7cyclictest1759273-21kworker/2:211:34:032
1753558996642,9cyclictest1755322-21kworker/2:009:27:342
1753550996641,21cyclictest16850irq/134-800000011:50:010
1753550996641,21cyclictest16850irq/134-800000011:50:010
1753558996539,8cyclictest1759273-21kworker/2:210:14:162
1753558996539,8cyclictest1759273-21kworker/2:210:14:162
175355399644,41cyclictest23-21ksoftirqd/107:25:021
175355099641,43cyclictest12-21ksoftirqd/012:08:250
175355099641,43cyclictest12-21ksoftirqd/012:08:250
1753558996343,5cyclictest1755322-21kworker/2:012:05:162
1753558996343,5cyclictest1755322-21kworker/2:012:05:162
1753558996339,9cyclictest1759273-21kworker/2:210:05:252
1753558996339,7cyclictest1759273-21kworker/2:210:16:592
1753558996338,5cyclictest1759273-21kworker/2:209:35:072
1753550996334,19cyclictest12-21ksoftirqd/009:55:020
1753558996129,7cyclictest1740790-21kworker/2:308:25:042
1753550996040,15cyclictest3467602-21systemd-journal12:20:010
1753558995939,5cyclictest1759273-21kworker/2:209:31:402
1753558995935,6cyclictest1755322-21kworker/2:010:50:002
1753553995937,14cyclictest1-21systemd09:40:001
175355399592,9cyclictest23-21ksoftirqd/110:45:021
1753550995923,31cyclictest1799730-21sh10:33:400
1753550995920,3cyclictest131rcu_preempt12:03:040
1753553995811,6cyclictest131rcu_preempt11:59:591
1753550995829,14cyclictest1852289-21ssh12:23:580
175355099582,30cyclictest131rcu_preempt10:23:230
1753558995749,5cyclictest1759273-21kworker/2:209:58:262
1753553995712,6cyclictest131rcu_preempt11:46:231
1753553995712,6cyclictest131rcu_preempt11:46:231
1753550995728,13cyclictest1801363-21ssh10:36:550
175355399566,6cyclictest131rcu_preempt12:15:011
1753550995635,18cyclictest1819668-21latency_hist11:15:030
1753550995628,13cyclictest1789688-21ssh10:12:350
1753550995628,13cyclictest1789688-21ssh10:12:340
1753550995628,13cyclictest1760522-21ssh09:11:250
1753550995627,14cyclictest1820149-21ssh11:16:120
1753550995626,3cyclictest131rcu_preempt09:30:010
1753550995625,15cyclictest1839017-21ssh11:55:570
1753550995618,33cyclictest16850irq/134-800000009:44:220
1753558995547,5cyclictest1759273-21kworker/2:210:01:452
1753558995546,6cyclictest1759273-21kworker/2:209:15:532
1753558995546,6cyclictest1759273-21kworker/2:209:15:532
1753558995546,6cyclictest1755322-21kworker/2:011:46:422
1753558995546,6cyclictest1755322-21kworker/2:011:46:422
1753553995537,12cyclictest275-21dbus-daemon09:25:001
1753553995514,6cyclictest131rcu_preempt09:00:011
1753553995514,6cyclictest131rcu_preempt09:00:011
1753550995528,12cyclictest1788285-21ssh10:09:510
1753550995526,3cyclictest131rcu_preempt11:51:270
175355099552,18cyclictest131rcu_preempt11:08:160
1753550995519,6cyclictest16850irq/134-800000012:27:550
1753550995519,31cyclictest16850irq/134-800000011:23:370
1753550995519,31cyclictest16850irq/134-800000010:19:180
1753550995518,32cyclictest16850irq/134-800000009:17:280
1753550995518,32cyclictest16850irq/134-800000009:17:270
175355099551,38cyclictest1810442-21ssh10:55:510
175355099551,35cyclictest131rcu_preempt11:39:420
175355099551,35cyclictest131rcu_preempt11:39:420
1753558995445,6cyclictest1759273-21kworker/2:211:00:112
1753558995431,8cyclictest30-21ksoftirqd/212:35:002
175355399545,6cyclictest131rcu_preempt10:09:571
1753553995432,11cyclictest309-21in:imuxsock08:15:011
1753553995415,14cyclictest23-21ksoftirqd/111:55:001
1753553995412,6cyclictest131rcu_preempt12:30:021
175355099542,17cyclictest1-21systemd10:00:000
1753550995418,32cyclictest16850irq/134-800000012:33:460
1753550995418,32cyclictest16850irq/134-800000011:40:190
1753550995418,31cyclictest16850irq/134-800000011:25:410
1753550995418,31cyclictest16850irq/134-800000011:25:410
1753550995418,31cyclictest16850irq/134-800000010:41:560
1753558995345,5cyclictest1759273-21kworker/2:211:25:112
1753558995345,5cyclictest1759273-21kworker/2:211:25:102
1753558995345,5cyclictest1755322-21kworker/2:011:56:582
1753558995345,5cyclictest1755322-21kworker/2:011:20:032
1753558995345,5cyclictest1755322-21kworker/2:010:43:132
1753558995345,5cyclictest1740790-21kworker/2:307:15:022
1753558995345,4cyclictest1755322-21kworker/2:009:46:572
1753558995345,4cyclictest1755322-21kworker/2:009:10:012
1753558995344,5cyclictest1759273-21kworker/2:212:19:222
1753558995321,21cyclictest30-21ksoftirqd/207:40:002
175355099535,3cyclictest131rcu_preempt12:14:000
1753550995318,6cyclictest16850irq/134-800000011:02:240
1753560995237,11cyclictest1-21systemd11:55:003
1753558995244,5cyclictest1755322-21kworker/2:011:50:062
1753558995244,5cyclictest1755322-21kworker/2:011:17:582
1753558995244,5cyclictest1755322-21kworker/2:010:28:072
1753558995244,5cyclictest1740790-21kworker/2:307:55:012
1753558995241,6cyclictest1759273-21kworker/2:209:43:352
1753558995238,6cyclictest1755322-21kworker/2:009:12:212
175355399526,24cyclictest23-21ksoftirqd/112:22:511
1753553995237,11cyclictest1-21systemd11:20:011
1753553995220,13cyclictest23-21ksoftirqd/107:30:011
1753553995217,5cyclictest131rcu_preempt09:45:021
175355399521,5cyclictest131rcu_preempt11:33:331
1753553995210,6cyclictest131rcu_preempt09:48:221
1753550995225,12cyclictest1765454-21ssh09:21:520
1753550995218,6cyclictest16850irq/134-800000009:33:410
1753550995218,30cyclictest16850irq/134-800000010:04:030
1753550995218,30cyclictest16850irq/134-800000009:36:050
1753550995217,31cyclictest16850irq/134-800000011:33:240
1753558995143,5cyclictest1759273-21kworker/2:212:35:542
1753558995143,5cyclictest1759273-21kworker/2:212:25:422
1753558995143,5cyclictest1759273-21kworker/2:211:42:172
1753558995142,6cyclictest1755322-21kworker/2:012:11:272
175355399515,8cyclictest131rcu_preempt11:00:011
175355399513,12cyclictest23-21ksoftirqd/111:30:001
175355399513,12cyclictest23-21ksoftirqd/111:30:001
175355399512,26cyclictest221rcuc/108:45:031
1753553995115,7cyclictest131rcu_preempt09:30:021
1753553995113,6cyclictest131rcu_preempt10:52:261
1753550995128,19cyclictest1778982-21cut09:50:020
1753550995115,32cyclictest16850irq/134-800000012:39:520
1753558995042,5cyclictest1759273-21kworker/2:212:24:022
1753558995042,5cyclictest1759273-21kworker/2:211:35:312
1753558995042,5cyclictest1759273-21kworker/2:211:35:312
1753558995042,5cyclictest1759273-21kworker/2:210:39:272
1753558995042,5cyclictest1759273-21kworker/2:210:33:402
1753558995040,7cyclictest1755322-21kworker/2:011:11:012
175355399506,7cyclictest23-21ksoftirqd/112:17:281
1753553995034,12cyclictest1-21systemd08:10:011
1753553995030,15cyclictest275-21dbus-daemon09:15:011
1753550995033,13cyclictest1-21systemd07:40:000
1753558994941,5cyclictest1759273-21kworker/2:209:21:112
1753558994941,5cyclictest1755322-21kworker/2:012:02:122
1753558994941,5cyclictest1755322-21kworker/2:011:05:052
1753558994941,5cyclictest1740790-21kworker/2:308:20:012
1753558994940,5cyclictest1755322-21kworker/2:010:22:462
175355899492,5cyclictest131rcu_preempt08:38:032
175355399499,5cyclictest131rcu_preempt07:40:001
175355399498,7cyclictest131rcu_preempt12:35:001
175355399491,9cyclictest23-21ksoftirqd/110:40:021
1753553994915,7cyclictest131rcu_preempt11:05:231
1753553994914,7cyclictest131rcu_preempt10:47:221
175355099494,39cyclictest1753995-21modprobe07:20:010
175355099494,39cyclictest1753995-21modprobe07:20:010
1753558994840,5cyclictest1755322-21kworker/2:009:02:462
1753558994840,5cyclictest1755322-21kworker/2:008:52:492
175355399487,11cyclictest23-21ksoftirqd/112:37:311
175355399486,9cyclictest23-21ksoftirqd/110:15:021
175355399486,9cyclictest23-21ksoftirqd/110:15:011
175355399482,5cyclictest131rcu_preempt11:21:081
1753553994816,6cyclictest131rcu_preempt12:10:001
1753553994816,6cyclictest131rcu_preempt12:09:591
1753553994813,11cyclictest23-21ksoftirqd/110:20:021
1753550994829,14cyclictest1-21systemd09:10:010
1753558994739,5cyclictest1740790-21kworker/2:308:01:352
1753558994739,5cyclictest1740790-21kworker/2:307:50:022
1753558994739,5cyclictest1740790-21kworker/2:307:43:342
1753558994739,5cyclictest1740790-21kworker/2:307:35:012
1753558994739,5cyclictest1740790-21kworker/2:307:35:002
1753558994738,5cyclictest1740790-21kworker/2:307:58:422
175355899472,6cyclictest131rcu_preempt09:00:012
175355899472,6cyclictest131rcu_preempt09:00:002
175355399479,5cyclictest131rcu_preempt10:20:451
175355399475,9cyclictest23-21ksoftirqd/111:04:031
175355399475,6cyclictest23-21ksoftirqd/108:25:001
175355399474,9cyclictest23-21ksoftirqd/107:50:011
175355399471,8cyclictest0-21swapper/109:16:021
175355399471,8cyclictest0-21swapper/109:16:021
175355399471,21cyclictest0-21swapper/108:00:021
175355399471,19cyclictest0-21swapper/110:29:121
1753553994711,5cyclictest131rcu_preempt10:01:471
1753558994638,5cyclictest1755322-21kworker/2:008:40:242
1753558994638,5cyclictest1740790-21kworker/2:308:27:062
1753558994637,7cyclictest30-21ksoftirqd/208:50:022
175355399465,9cyclictest23-21ksoftirqd/109:32:501
1753553994615,9cyclictest23-21ksoftirqd/109:04:321
1753553994615,5cyclictest131rcu_preempt09:56:551
1753553994610,5cyclictest131rcu_preempt11:42:401
1753550994639,4cyclictest16950irq/224-800000008:50:000
1753550994638,5cyclictest16950irq/224-800000009:00:010
1753550994638,5cyclictest16950irq/224-800000009:00:010
175356099454,4cyclictest131rcu_preempt11:05:023
1753560994531,10cyclictest3466785-21systemd-udevd09:47:593
1753560994530,11cyclictest1833862-21date11:45:023
1753560994529,12cyclictest275-21dbus-daemon09:05:013
1753558994537,5cyclictest1740790-21kworker/2:308:13:272
1753553994535,7cyclictest23-21ksoftirqd/107:20:031
1753553994535,7cyclictest23-21ksoftirqd/107:20:021
175355399451,5cyclictest131rcu_preempt11:11:461
1753553994512,6cyclictest131rcu_preempt08:55:001
1753560994429,11cyclictest0-21swapper/310:05:023
1753558994436,5cyclictest1740790-21kworker/2:307:15:542
1753558994436,5cyclictest1740790-21kworker/2:307:15:532
175355399448,6cyclictest131rcu_preempt09:52:551
175355399443,11cyclictest23-21ksoftirqd/109:10:011
1753553994410,6cyclictest131rcu_preempt08:35:011
1753553994410,6cyclictest131rcu_preempt08:35:011
175355099441,39cyclictest3467602-21systemd-journal07:50:220
175356099438,9cyclictest37-21ksoftirqd/311:20:293
175356099434,3cyclictest131rcu_preempt09:21:263
175356099432,8cyclictest37-21ksoftirqd/308:39:213
1753560994327,12cyclictest1-21systemd07:55:013
1753560994325,13cyclictest275-21dbus-daemon07:20:013
1753560994325,13cyclictest275-21dbus-daemon07:20:003
1753558994335,5cyclictest1740790-21kworker/2:307:29:442
175355399436,12cyclictest23-21ksoftirqd/108:45:011
175355399434,29cyclictest1758197-21latency_hist08:39:591
1753553994329,10cyclictest275-21dbus-daemon07:35:021
1753553994329,10cyclictest275-21dbus-daemon07:35:021
1753553994327,9cyclictest309-21in:imuxsock07:55:011
1753553994318,15cyclictest0-21swapper/108:19:591
1753553994312,6cyclictest131rcu_preempt08:30:011
1753550994326,13cyclictest1757613-21sshd08:27:220
1753550994323,15cyclictest275-21dbus-daemon07:15:000
175356099429,3cyclictest131rcu_preempt10:25:023
1753560994232,6cyclictest1834463-21sshd11:46:233
1753560994232,6cyclictest1834463-21sshd11:46:223
1753560994227,13cyclictest37-21ksoftirqd/310:46:303
1753560994227,11cyclictest1-21systemd10:30:013
175356099422,22cyclictest0-21swapper/311:07:223
175356099421,37cyclictest37-21ksoftirqd/309:35:553
1753558994234,5cyclictest1740790-21kworker/2:308:32:202
1753558994234,5cyclictest1740790-21kworker/2:308:32:202
1753553994225,13cyclictest275-21dbus-daemon08:05:011
1753550994229,9cyclictest1757113-21cat08:20:000
1753550994226,12cyclictest1758950-21sshd08:52:220
175356099418,5cyclictest131rcu_preempt12:05:003
175356099416,7cyclictest37-21ksoftirqd/311:13:493
175356099416,7cyclictest37-21ksoftirqd/309:17:113
175356099416,7cyclictest37-21ksoftirqd/309:17:113
175356099414,4cyclictest131rcu_preempt11:17:493
1753560994130,8cyclictest1781555-21sshd09:55:233
1753560994127,11cyclictest3467602-21systemd-journal07:40:223
1753560994126,11cyclictest275-21dbus-daemon09:00:023
1753560994126,11cyclictest275-21dbus-daemon09:00:023
1753560994126,11cyclictest275-21dbus-daemon07:35:013
1753560994126,11cyclictest275-21dbus-daemon07:35:003
1753560994125,13cyclictest37-21ksoftirqd/312:33:393
175356099411,4cyclictest131rcu_preempt12:30:013
1753558994124,12cyclictest30-21ksoftirqd/207:20:032
175355399412,5cyclictest131rcu_preempt07:45:021
175355399412,10cyclictest23-21ksoftirqd/107:15:021
1753550994131,7cyclictest12-21ksoftirqd/007:30:020
175356099409,6cyclictest131rcu_preempt09:11:183
175356099404,7cyclictest131rcu_preempt10:53:333
175356099404,3cyclictest131rcu_preempt11:56:243
175356099404,3cyclictest131rcu_preempt10:38:003
175356099404,3cyclictest131rcu_preempt10:31:393
175356099403,3cyclictest131rcu_preempt10:41:163
1753560994027,10cyclictest37-21ksoftirqd/312:15:333
1753560994027,10cyclictest1771727-21wc09:35:023
1753560994026,12cyclictest37-21ksoftirqd/309:27:293
175356099401,9cyclictest1810846-21ssh10:56:433
1753560994018,18cyclictest131rcu_preempt12:13:313
175356099401,4cyclictest0-21swapper/311:33:483
175355899401,6cyclictest131rcu_preempt08:10:022
1753550994023,13cyclictest1756322-21sshd08:03:210
175356099395,4cyclictest131rcu_preempt11:28:513
175356099395,4cyclictest131rcu_preempt11:28:503
175356099394,4cyclictest131rcu_preempt10:12:033
175356099394,4cyclictest131rcu_preempt10:12:023
175356099394,3cyclictest131rcu_preempt12:37:223
175356099393,4cyclictest131rcu_preempt12:05:543
175356099393,4cyclictest131rcu_preempt12:05:543
175356099393,4cyclictest131rcu_preempt11:38:353
175356099393,4cyclictest131rcu_preempt11:38:343
175356099393,4cyclictest131rcu_preempt09:51:023
175356099393,3cyclictest131rcu_preempt07:25:003
175356099392,4cyclictest131rcu_preempt10:18:113
175356099392,3cyclictest131rcu_preempt08:15:013
1753550993926,10cyclictest3467718-21lldpd08:43:060
1753550993925,11cyclictest1754244-21sshd07:21:210
175355099391,5cyclictest1756861-21munin-run08:15:000
175356099383,3cyclictest131rcu_preempt08:20:023
1753560993828,7cyclictest37-21ksoftirqd/309:10:023
1753560993828,7cyclictest37-21ksoftirqd/308:45:003
1753560993826,9cyclictest1776237-21sshd09:44:353
1753560993826,9cyclictest1757543-21sed08:25:053
175356099382,4cyclictest131rcu_preempt12:24:063
175356099382,4cyclictest131rcu_preempt10:08:363
1753560993824,11cyclictest1-21systemd08:30:013
175356099381,9cyclictest3467602-21systemd-journal07:10:013
1753553993828,8cyclictest23-21ksoftirqd/107:10:021
175355099383,6cyclictest3467602-21systemd-journal08:22:390
1753550993824,11cyclictest275-21dbus-daemon08:38:020
175356099371,3cyclictest131rcu_preempt08:10:013
1753550993727,7cyclictest0-21swapper/007:32:210
1753550993727,7cyclictest0-21swapper/007:32:210
1753550993727,6cyclictest1756073-21munin-run08:00:000
175356099362,3cyclictest131rcu_preempt08:50:013
175356099362,3cyclictest131rcu_preempt08:00:013
1753550993621,11cyclictest1753693-21latency_hist07:10:020
175355099361,5cyclictest1756712-21sed08:10:020
175356099353,3cyclictest131rcu_preempt07:50:013
1753560993524,8cyclictest37-21ksoftirqd/308:05:013
175356099351,3cyclictest131rcu_preempt07:40:013
175356099351,3cyclictest131rcu_preempt07:30:013
1753550993510,6cyclictest131rcu_preempt07:50:010
175356099342,4cyclictest131rcu_preempt08:35:013
175356099342,4cyclictest131rcu_preempt08:35:013
175356099342,3cyclictest131rcu_preempt08:55:023
175356099342,3cyclictest131rcu_preempt07:10:023
175355099341,5cyclictest1755285-21modprobe07:42:210
175355099341,4cyclictest0-21swapper/009:00:360
175355899311,7cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional