You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 09:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Wed May 06, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
220206422300,8sleep20-21swapper/219:05:262
222184221280,2sleep20-21swapper/220:00:172
226699721210,3sleep20-21swapper/222:10:012
221982521080,2sleep30-21swapper/319:55:133
228180021030,2sleep03056-21PK-Backend22:50:170
220131727846,22sleep10-21swapper/119:05:141
220260727450,18sleep00-21swapper/019:08:020
220128127140,25sleep30-21swapper/319:05:083
22465092530,4sleep30-21swapper/321:10:153
22647472490,2sleep00-21swapper/022:00:360
139099310,5rtkit-daemon1389-21rtkit-daemon19:35:121
2202893992712,13cyclictest2303568-21unin-run23:55:000
139099270,4rtkit-daemon1389-21rtkit-daemon20:08:310
22340482260,2sleep0161rcuc/020:35:150
2202893992514,9cyclictest2305305-40ogrotate00:00:000
2202900992417,5cyclictest2300059-21cron23:45:002
2202900992417,5cyclictest2300059-21cron23:45:002
2202893992416,6cyclictest436-20systemd-journal22:50:000
220290099232,19cyclictest2238573-21apt20:50:002
139099230,4rtkit-daemon1389-21rtkit-daemon19:41:000
139099230,16rtkit-daemon3056-21PK-Backend22:10:353
2202903992220,1cyclictest3056-21PK-Backend22:53:113
2202903992220,1cyclictest3056-21PK-Backend21:40:243
2202903992220,1cyclictest3056-21PK-Backend20:00:283
2202903992220,1cyclictest0-21swapper/323:23:263
2202903992219,2cyclictest3056-21PK-Backend22:09:313
2202903992219,2cyclictest3056-21PK-Backend00:19:163
2202903992219,2cyclictest0-21swapper/320:37:583
2202893992214,6cyclictest2211650-21users19:30:230
220289399220,20cyclictest0-21swapper/020:30:170
139099220,8rtkit-daemon0-21swapper/221:43:152
139099220,15rtkit-daemon0-21swapper/223:45:162
22897592210,2sleep20-21swapper/223:15:012
22761682210,4chrt3056-21PK-Backend22:35:163
220290399212,4cyclictest2248630-21python321:15:213
2202903992119,1cyclictest3056-21PK-Backend22:47:303
2202903992119,1cyclictest0-21swapper/322:56:013
2202903992119,1cyclictest0-21swapper/320:44:203
2202903992118,2cyclictest3056-21PK-Backend23:16:573
2202903992118,2cyclictest3056-21PK-Backend21:50:383
2202903992118,2cyclictest3056-21PK-Backend19:41:343
2202903992113,6cyclictest2238032-21cat20:45:213
220290399210,20cyclictest3056-21PK-Backend22:41:483
220290399210,20cyclictest3056-21PK-Backend22:41:483
220290399210,20cyclictest0-21swapper/322:30:063
220290399210,20cyclictest0-21swapper/300:26:393
220290399210,19cyclictest3056-21PK-Backend22:15:053
220290399210,19cyclictest3056-21PK-Backend20:05:543
220290399210,19cyclictest3056-21PK-Backend19:34:453
220290399210,19cyclictest3056-21PK-Backend00:04:423
2202900992119,1cyclictest0-21swapper/220:11:172
2202900992111,9cyclictest2252706-21apt-get21:30:012
2202897992119,1cyclictest0-21swapper/122:19:401
2202897992111,8cyclictest26-21ksoftirqd/100:19:231
2202893992115,5cyclictest883-21dbus-daemon21:50:000
2202893992113,6cyclictest883-21dbus-daemon23:40:580
2202893992113,6cyclictest883-21dbus-daemon23:40:580
220289399210,20cyclictest0-21swapper/020:04:550
220289399210,1cyclictest0-21swapper/023:29:020
220289399210,1cyclictest0-21swapper/020:12:250
139099210,18rtkit-daemon953-21gmain20:06:171
139099210,15rtkit-daemon3056-21PK-Backend23:03:083
139099210,14rtkit-daemon3056-21PK-Backend21:45:063
22050122200,4sleep23056-21PK-Backend19:15:022
2202903992018,1cyclictest3056-21PK-Backend23:11:163
2202903992018,1cyclictest3056-21PK-Backend19:46:073
2202903992018,1cyclictest0-21swapper/321:04:013
2202903992012,6cyclictest2309177-21expr00:10:123
2202903992012,6cyclictest2227562-21python320:15:193
220290399200,19cyclictest3056-21PK-Backend23:59:353
220290399200,19cyclictest3056-21PK-Backend23:42:593
220290399200,19cyclictest3056-21PK-Backend23:42:593
220290399200,19cyclictest3056-21PK-Backend23:07:233
220290399200,19cyclictest0-21swapper/322:20:393
220290399200,19cyclictest0-21swapper/321:29:573
220290399200,19cyclictest0-21swapper/321:07:463
220290399200,19cyclictest0-21swapper/320:54:553
220290399200,19cyclictest0-21swapper/319:50:333
2202900992018,1cyclictest0-21swapper/222:38:382
2202900992018,1cyclictest0-21swapper/220:33:062
2202900992018,1cyclictest0-21swapper/220:08:252
2202900992018,1cyclictest0-21swapper/200:38:302
2202900992018,1cyclictest0-21swapper/200:10:382
2202900992013,6cyclictest883-21dbus-daemon00:31:002
2202900992013,6cyclictest2303536-21cron23:55:002
2202900992010,7cyclictest883-21dbus-daemon00:25:582
220290099200,19cyclictest0-21swapper/222:40:272
220290099200,19cyclictest0-21swapper/222:40:272
220290099200,19cyclictest0-21swapper/222:10:472
220290099200,19cyclictest0-21swapper/220:50:362
220290099200,19cyclictest0-21swapper/220:23:122
220290099200,19cyclictest0-21swapper/219:44:462
220290099200,19cyclictest0-21swapper/219:21:422
220290099200,19cyclictest0-21swapper/219:10:252
2202897992018,1cyclictest0-21swapper/121:48:031
2202897992012,6cyclictest883-21dbus-daemon23:45:001
2202897992012,6cyclictest883-21dbus-daemon23:44:591
220289799201,17cyclictest2252706-21apt-get21:30:001
220289799200,19cyclictest0-21swapper/100:23:511
220289799200,11cyclictest23010802ntp_states23:45:211
2202893992018,1cyclictest0-21swapper/023:49:300
2202893992018,1cyclictest0-21swapper/023:22:330
2202893992018,1cyclictest0-21swapper/021:19:040
2202893992018,1cyclictest0-21swapper/020:58:080
2202893992018,1cyclictest0-21swapper/020:21:110
2202893992018,1cyclictest0-21swapper/000:26:140
2202893992013,5cyclictest13-21ksoftirqd/022:25:120
220289399200,1cyclictest3056-21PK-Backend19:50:220
220289399200,19cyclictest3056-21PK-Backend21:30:460
220289399200,19cyclictest3056-21PK-Backend20:40:240
220289399200,19cyclictest0-21swapper/023:13:000
220289399200,19cyclictest0-21swapper/023:01:240
220289399200,19cyclictest0-21swapper/021:21:270
220289399200,19cyclictest0-21swapper/019:13:590
220289399200,18cyclictest0-21swapper/022:12:580
139099200,14rtkit-daemon0-21swapper/223:03:592
139099200,14rtkit-daemon0-21swapper/221:55:502
139099200,13rtkit-daemon3056-21PK-Backend23:49:303
139099200,13rtkit-daemon0-21swapper/200:16:212
22651622190,4sleep33056-21PK-Backend22:05:013
22340102190,3sleep20-21swapper/220:35:132
22103162190,2sleep12210314-21sleep19:30:011
220290399192,15cyclictest3056-21PK-Backend00:07:473
2202903991917,1cyclictest3056-21PK-Backend19:11:473
2202903991913,4cyclictest2293557-21tr23:25:123
2202903991912,5cyclictest2296580-21cron23:34:593
220290399190,7cyclictest2254304-21/usr/sbin/munin21:30:213
220290399190,1cyclictest3056-21PK-Backend23:50:223
220290399190,1cyclictest3056-21PK-Backend22:26:273
220290399190,1cyclictest3056-21PK-Backend21:39:363
220290399190,1cyclictest3056-21PK-Backend20:34:473
220290399190,1cyclictest3056-21PK-Backend20:10:343
220290399190,1cyclictest3056-21PK-Backend19:37:023
220290399190,1cyclictest3056-21PK-Backend00:34:503
220290399190,1cyclictest0-21swapper/320:21:563
220290399190,18cyclictest0-21swapper/320:58:073
220290099195,12cyclictest2265110-21munin-run22:05:002
2202900991917,1cyclictest0-21swapper/223:38:412
2202900991917,1cyclictest0-21swapper/223:31:452
2202900991917,1cyclictest0-21swapper/222:45:072
2202900991917,1cyclictest0-21swapper/221:45:112
2202900991917,1cyclictest0-21swapper/221:23:282
2202900991917,1cyclictest0-21swapper/220:16:572
2202900991917,1cyclictest0-21swapper/219:59:192
2202900991917,1cyclictest0-21swapper/219:46:012
2202900991914,3cyclictest978-21snmpd20:41:012
2202900991911,6cyclictest2246507-21awk21:10:152
220290099190,1cyclictest0-21swapper/220:56:242
220290099190,1cyclictest0-21swapper/219:27:022
220290099190,1cyclictest0-21swapper/200:04:302
220290099190,18cyclictest0-21swapper/223:16:582
220290099190,18cyclictest0-21swapper/222:25:062
220290099190,18cyclictest0-21swapper/221:06:042
220290099190,18cyclictest0-21swapper/219:35:542
2202897991918,1cyclictest0-21swapper/119:11:081
2202897991917,1cyclictest0-21swapper/121:30:531
2202897991917,1cyclictest0-21swapper/119:44:131
2202897991913,5cyclictest2275035-21cat22:30:221
2202897991913,4cyclictest26-21ksoftirqd/121:40:551
2202897991912,5cyclictest2308026-21ntp_states00:05:201
2202897991910,7cyclictest2216148-21df19:45:131
2202897991910,7cyclictest2206238-21python319:15:211
220289799190,18cyclictest0-21swapper/121:35:191
2202893991917,1cyclictest0-21swapper/022:55:430
2202893991917,1cyclictest0-21swapper/022:36:030
2202893991917,1cyclictest0-21swapper/021:11:330
2202893991917,1cyclictest0-21swapper/020:53:220
220289399190,1cyclictest0-21swapper/023:30:040
220289399190,1cyclictest0-21swapper/023:06:180
220289399190,1cyclictest0-21swapper/022:42:180
220289399190,1cyclictest0-21swapper/022:42:180
220289399190,1cyclictest0-21swapper/022:05:480
220289399190,1cyclictest0-21swapper/021:55:270
220289399190,1cyclictest0-21swapper/021:41:480
220289399190,1cyclictest0-21swapper/021:28:230
220289399190,1cyclictest0-21swapper/020:25:440
220289399190,1cyclictest0-21swapper/019:48:320
220289399190,18cyclictest0-21swapper/023:38:490
220289399190,18cyclictest0-21swapper/022:30:350
220289399190,18cyclictest0-21swapper/021:09:580
220289399190,18cyclictest0-21swapper/019:29:470
139099190,15rtkit-daemon2202888-21cyclictest20:16:380
139099190,13rtkit-daemon0-21swapper/222:30:352
22381632180,3sleep03056-21PK-Backend20:45:220
2202903991816,1cyclictest0-21swapper/323:36:373
2202903991815,2cyclictest3056-21PK-Backend19:24:033
2202903991812,5cyclictest42-21ksoftirqd/320:25:203
2202903991811,5cyclictest2249483-21cat21:20:113
2202900991816,1cyclictest0-21swapper/221:03:072
2202900991816,1cyclictest0-21swapper/219:50:272
2202900991816,1cyclictest0-21swapper/219:33:372
2202900991815,2cyclictest0-21swapper/220:28:332
2202900991813,4cyclictest978-21snmpd22:56:502
2202900991811,5cyclictest2252795-21apt-key21:30:002
220290099180,1cyclictest0-21swapper/200:06:322
220289799187,9cyclictest2280764-21unin-run22:50:001
220289799182,12cyclictest2284727-21apt23:00:121
2202897991816,1cyclictest0-21swapper/120:37:061
2202897991813,3cyclictest2288834-21needreboot23:10:181
2202897991812,4cyclictest26-21ksoftirqd/123:05:191
2202897991812,4cyclictest2305305-40ogrotate00:00:011
2202897991812,4cyclictest2273761-21cron22:30:001
2202897991811,5cyclictest2309947-21cat00:10:211
220289799180,1cyclictest0-21swapper/120:00:291
2202893991816,1cyclictest0-21swapper/019:55:490
2202893991816,1cyclictest0-21swapper/019:23:250
2202893991812,4cyclictest2306823-21sshd00:02:390
2202893991811,5cyclictest2319625-21wc00:40:000
2202893991811,5cyclictest2213031-21ntpq19:35:180
2202893991811,5cyclictest1894998-21snapd00:19:290
2202893991810,6cyclictest2290286-21df23:15:130
23150002170,4sleep10-21swapper/100:25:171
2202900991715,1cyclictest0-21swapper/223:26:172
2202900991715,1cyclictest0-21swapper/223:08:402
2202900991715,1cyclictest0-21swapper/222:15:332
2202900991715,1cyclictest0-21swapper/221:51:062
2202897991715,1cyclictest3056-21PK-Backend21:03:561
2202897991715,1cyclictest3056-21PK-Backend19:51:241
2202897991715,1cyclictest0-21swapper/122:21:501
2202897991715,1cyclictest0-21swapper/122:05:141
2202897991715,1cyclictest0-21swapper/121:21:341
2202897991715,1cyclictest0-21swapper/100:35:191
2202897991715,1cyclictest0-21swapper/100:33:311
2202897991712,3cyclictest2302199-21cut23:50:121
2202897991712,3cyclictest2228576-21cpuspeed_turbos20:20:141
2202897991711,5cyclictest2295846-21awk23:30:171
2202897991710,5cyclictest2263457-21ls22:00:011
2202893991715,1cyclictest0-21swapper/021:53:510
2202893991712,3cyclictest2256422-21cron21:40:000
2202893991710,6cyclictest2317228-21sed00:30:220
139099170,8rtkit-daemon2256041-21vmstat21:35:242
2202900991614,1cyclictest0-21swapper/221:15:102
2202900991613,2cyclictest2271436-21smartctl22:20:202
220290099160,15cyclictest3056-21PK-Backend23:23:272
220289799169,6cyclictest2276163-21df22:35:151
220289799169,6cyclictest2207860-21open_files19:20:191
220289799168,6cyclictest2290712-21which23:15:201
2202897991614,1cyclictest0-21swapper/122:58:201
2202897991614,1cyclictest0-21swapper/119:30:281
2202897991611,3cyclictest2282200-21threads22:50:221
2202897991611,3cyclictest2247502-21taskset21:15:001
2202897991611,3cyclictest2225413-21iwlist20:10:161
2202897991610,5cyclictest2297306-21awk23:35:161
220289399169,5cyclictest2308256-21sed00:05:230
220289399169,5cyclictest2269544-21ntpq22:15:190
2202893991614,1cyclictest0-21swapper/000:14:310
139099160,13rtkit-daemon1317-21systemd20:28:451
2202903991513,1cyclictest3056-21PK-Backend19:29:243
2202903991512,2cyclictest2314229-21cron00:25:003
2202903991510,3cyclictest2318885-21grep00:35:223
220290099159,4cyclictest34-21ksoftirqd/200:00:012
2202900991511,3cyclictest2282724-21idleruntime-cro22:55:002
2202900991510,3cyclictest2314244-21munin-run00:25:012
220289799159,4cyclictest2305681-21grep00:00:141
220289799159,4cyclictest2293588-21ls23:25:131
220289799159,4cyclictest2278246-21ntpq22:40:181
220289799159,4cyclictest2278246-21ntpq22:40:181
220289799159,4cyclictest2264214-21load22:00:171
220289799158,5cyclictest2260915-21ntp_states21:50:191
220289799158,5cyclictest2238563-21cron20:50:011
220289799158,5cyclictest2226930-21expr20:15:111
220289799153,8cyclictest2267109-21apt22:10:121
2202897991513,1cyclictest0-21swapper/121:05:521
2202897991510,4cyclictest2239034-21tr20:50:121
2202897991510,3cyclictest2232122-21cstates20:30:121
2202897991510,3cyclictest1095-21runrttasks21:19:181
220289799150,14cyclictest3056-21PK-Backend20:55:381
220289399158,5cyclictest2313176-21which00:20:170
2202893991511,3cyclictest2243765-21sh21:05:000
2202893991510,3cyclictest2204886-21if_enp2s019:15:010
2202903991411,2cyclictest3056-21PK-Backend19:15:553
220289799142,2cyclictest141rcu_preempt19:57:101
2202897991412,1cyclictest3056-21PK-Backend20:41:461
220289799140,13cyclictest0-21swapper/123:24:161
220289399143,7cyclictest2270576-21apt22:20:110
2202903991311,1cyclictest3056-21PK-Backend21:55:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional