You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 08:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Apr 29, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
244909222800,9sleep20-21swapper/219:05:162
244946222230,22sleep10-21swapper/119:05:231
245703421130,4sleep1899-21polkitd19:15:041
245703421130,4sleep1899-21polkitd19:15:031
253642621070,2sleep00-21swapper/022:35:150
244907921070,5sleep00-21swapper/019:05:140
247214921020,3sleep00-21swapper/019:35:150
1390991020,87rtkit-daemon0-21swapper/319:06:033
24638522670,4sleep23056-21PK-Backend19:16:092
24638522670,4sleep23056-21PK-Backend19:16:082
25222862450,2sleep00-21swapper/021:55:220
25415492410,3sleep30-21swapper/322:50:143
2450341993925,4cyclictest34-21ksoftirqd/219:51:242
25553312350,3sleep32555330-21cron23:30:003
24673342320,3sleep00-21swapper/019:20:190
2450341993026,3cyclictest34-21ksoftirqd/219:14:022
2450337992923,4cyclictest26-21ksoftirqd/119:15:001
2450341992824,3cyclictest331rcuc/220:05:252
2450337992812,14cyclictest2478536-21unin-run19:55:001
24759132260,3sleep30-21swapper/319:45:163
245034399268,16cyclictest922-21systemd-logind19:15:073
245034399268,16cyclictest922-21systemd-logind19:15:073
245034199264,6cyclictest34-21ksoftirqd/223:50:132
25553182250,3sleep20-21swapper/223:29:552
2450337992519,4cyclictest436-20systemd-journal23:39:551
245033299250,10cyclictest2469767-21perf19:34:550
25334692240,3sleep10-21swapper/122:25:221
2450343992411,11cyclictest886-21NetworkManager19:14:093
245033799240,22cyclictest2473716-21cstates19:40:131
2450332992419,3cyclictest2579879-21cron00:40:000
2450332992416,6cyclictest883-21dbus-daemon19:16:090
2450332992416,6cyclictest883-21dbus-daemon19:16:090
139099240,21rtkit-daemon1997-21gvfs-afc-volume21:17:450
245034399238,13cyclictest2482684-21systemd20:05:133
2450343992314,7cyclictest2558081-21ntp_states23:35:193
2450343992314,6cyclictest1095-21runrttasks22:25:123
2450341992321,1cyclictest0-21swapper/221:03:022
2450341992321,1cyclictest0-21swapper/219:22:572
245034199230,7cyclictest2501591-21sshd20:58:512
245034199230,21cyclictest2510686-21apt21:25:132
2450337992317,4cyclictest899-21polkitd20:05:141
245033799230,21cyclictest3056-21PK-Backend00:11:331
139099230,16rtkit-daemon3056-21PK-Backend22:40:093
245034399225,14cyclictest2488191-21wc20:20:183
2450343992220,1cyclictest0-21swapper/323:21:203
2450343992220,1cyclictest0-21swapper/322:23:473
2450343992217,4cyclictest2511003-21cstates21:25:143
2450343992216,4cyclictest2500943-21ntp_kernel_err20:55:193
245034399220,20cyclictest0-21swapper/319:40:303
2450341992220,1cyclictest0-21swapper/220:43:282
2450341992220,1cyclictest0-21swapper/220:29:092
2450341992220,1cyclictest0-21swapper/220:04:142
2450341992219,2cyclictest331rcuc/220:17:262
2450341992219,2cyclictest331rcuc/220:17:262
2450341992218,3cyclictest331rcuc/223:34:512
2450341992218,3cyclictest331rcuc/223:22:442
245034199221,7cyclictest34-21ksoftirqd/221:05:162
2450341992215,5cyclictest34-21ksoftirqd/222:50:132
2450337992220,1cyclictest3056-21PK-Backend21:37:141
2450337992220,1cyclictest3056-21PK-Backend19:48:461
2450337992219,2cyclictest3056-21PK-Backend23:02:041
245033299220,21cyclictest0-21swapper/021:30:530
139099220,16rtkit-daemon0-21swapper/221:36:002
139099220,15rtkit-daemon3056-21PK-Backend23:45:443
139099220,14rtkit-daemon3056-21PK-Backend22:37:073
24891342210,5sleep13056-21PK-Backend20:25:001
245034399219,11cyclictest2567992-21/usr/sbin/munin00:05:203
245034399219,11cyclictest2567992-21/usr/sbin/munin00:05:203
2450343992119,1cyclictest3056-21PK-Backend20:00:033
2450343992119,1cyclictest0-21swapper/323:08:493
2450343992119,1cyclictest0-21swapper/322:57:003
2450343992119,1cyclictest0-21swapper/321:44:403
2450343992119,1cyclictest0-21swapper/320:52:013
2450343992119,1cyclictest0-21swapper/320:45:403
2450343992119,1cyclictest0-21swapper/320:43:433
2450343992119,1cyclictest0-21swapper/300:27:043
2450343992119,1cyclictest0-21swapper/300:13:113
2450343992118,2cyclictest3056-21PK-Backend00:22:103
245034399210,20cyclictest0-21swapper/320:37:083
245034399210,20cyclictest0-21swapper/320:14:163
245034399210,20cyclictest0-21swapper/300:34:483
245034399210,19cyclictest3056-21PK-Backend22:30:233
245034399210,19cyclictest0-21swapper/300:15:143
245034199214,10cyclictest34-21ksoftirqd/221:30:122
2450341992119,1cyclictest0-21swapper/223:38:222
2450341992119,1cyclictest0-21swapper/222:41:372
2450341992119,1cyclictest0-21swapper/222:21:502
2450341992119,1cyclictest0-21swapper/220:14:352
2450341992119,1cyclictest0-21swapper/219:27:442
2450341992118,2cyclictest331rcuc/220:36:472
2450341992117,3cyclictest331rcuc/223:04:482
2450341992117,3cyclictest331rcuc/222:56:382
2450341992117,3cyclictest331rcuc/222:49:122
2450341992117,3cyclictest331rcuc/221:22:382
2450341992117,3cyclictest331rcuc/221:10:032
2450341992117,3cyclictest331rcuc/219:59:172
2450341992113,6cyclictest2517248-21sshd21:43:512
2450341992111,2cyclictest141rcu_preempt22:33:542
245034199210,20cyclictest0-21swapper/223:19:222
245034199210,20cyclictest0-21swapper/200:24:112
245034199210,20cyclictest0-21swapper/200:11:072
2450337992119,1cyclictest3056-21PK-Backend22:52:511
2450337992119,1cyclictest3056-21PK-Backend22:17:561
2450337992119,1cyclictest3056-21PK-Backend22:00:051
2450337992119,1cyclictest3056-21PK-Backend21:52:141
2450337992119,1cyclictest3056-21PK-Backend21:20:101
2450337992119,1cyclictest3056-21PK-Backend20:37:111
2450337992119,1cyclictest0-21swapper/122:42:371
2450337992119,1cyclictest0-21swapper/121:02:541
2450337992119,1cyclictest0-21swapper/100:20:321
2450337992118,2cyclictest3056-21PK-Backend21:10:171
2450337992118,2cyclictest3056-21PK-Backend21:08:551
2450337992118,2cyclictest3056-21PK-Backend00:30:181
2450337992118,2cyclictest3056-21PK-Backend00:26:261
245033799210,20cyclictest3056-21PK-Backend23:57:331
245033799210,20cyclictest0-21swapper/123:40:301
245033799210,19cyclictest3056-21PK-Backend22:05:461
245033799210,19cyclictest3056-21PK-Backend21:59:171
245033799210,19cyclictest3056-21PK-Backend20:58:131
245033799210,19cyclictest3056-21PK-Backend00:05:511
245033799210,19cyclictest3056-21PK-Backend00:05:511
2450332992112,7cyclictest2528280-21cron22:15:010
245033299210,20cyclictest0-21swapper/022:27:310
245033299210,20cyclictest0-21swapper/020:20:160
245033299210,20cyclictest0-21swapper/019:13:240
139099210,15rtkit-daemon3056-21PK-Backend19:25:173
245034399208,9cyclictest42-21ksoftirqd/322:00:113
2450343992018,1cyclictest3056-21PK-Backend19:52:533
2450343992018,1cyclictest0-21swapper/323:57:573
2450343992018,1cyclictest0-21swapper/323:14:373
2450343992018,1cyclictest0-21swapper/322:06:173
2450343992018,1cyclictest0-21swapper/321:51:433
2450343992018,1cyclictest0-21swapper/320:32:353
2450343992018,1cyclictest0-21swapper/320:16:263
2450343992018,1cyclictest0-21swapper/320:16:263
2450343992018,1cyclictest0-21swapper/300:36:503
2450343992018,1cyclictest0-21swapper/300:00:473
2450343992012,6cyclictest2509696-21ntp_states21:20:193
245034399200,19cyclictest3056-21PK-Backend21:00:333
245034399200,19cyclictest0-21swapper/323:50:273
245034399200,19cyclictest0-21swapper/323:32:083
245034399200,19cyclictest0-21swapper/322:45:583
245034399200,19cyclictest0-21swapper/321:48:473
245034399200,19cyclictest0-21swapper/321:39:133
245034399200,19cyclictest0-21swapper/321:31:493
245034399200,19cyclictest0-21swapper/319:37:533
2450341992018,1cyclictest0-21swapper/223:40:252
2450341992018,1cyclictest0-21swapper/222:35:492
2450341992018,1cyclictest0-21swapper/222:25:492
2450341992018,1cyclictest0-21swapper/221:53:172
2450341992018,1cyclictest0-21swapper/220:30:442
2450341992018,1cyclictest0-21swapper/200:16:142
2450341992018,1cyclictest0-21swapper/200:09:042
2450341992018,1cyclictest0-21swapper/200:09:042
2450341992017,2cyclictest331rcuc/222:08:542
2450341992017,2cyclictest331rcuc/221:58:322
2450341992017,2cyclictest331rcuc/221:18:012
2450341992016,3cyclictest331rcuc/223:12:422
2450341992016,3cyclictest331rcuc/222:12:392
2450341992016,3cyclictest331rcuc/200:36:462
245034199200,6cyclictest2497114-21/usr/sbin/munin20:45:222
245034199200,19cyclictest0-21swapper/219:48:192
245034199200,19cyclictest0-21swapper/200:26:142
2450337992018,1cyclictest3056-21PK-Backend23:07:521
2450337992018,1cyclictest3056-21PK-Backend22:57:031
2450337992018,1cyclictest3056-21PK-Backend20:17:311
2450337992018,1cyclictest3056-21PK-Backend20:17:311
2450337992018,1cyclictest0-21swapper/119:27:491
2450337992017,2cyclictest3056-21PK-Backend21:30:381
2450337992017,2cyclictest3056-21PK-Backend21:29:161
2450337992013,5cyclictest2482031-21cron20:05:001
245033799200,2cyclictest2490673-21chrt20:28:531
245033799200,1cyclictest3056-21PK-Backend19:36:001
245033799200,19cyclictest3056-21PK-Backend20:52:251
245033799200,19cyclictest3056-21PK-Backend20:14:001
245033799200,19cyclictest0-21swapper/123:50:041
245033799200,19cyclictest0-21swapper/122:36:011
245033799200,18cyclictest0-21swapper/100:17:271
2450332992018,1cyclictest0-21swapper/023:49:170
2450332992018,1cyclictest0-21swapper/023:42:340
2450332992018,1cyclictest0-21swapper/020:00:020
245033299200,1cyclictest0-21swapper/020:34:360
245033299200,19cyclictest0-21swapper/023:21:250
245033299200,19cyclictest0-21swapper/022:55:570
245033299200,19cyclictest0-21swapper/022:51:440
245033299200,19cyclictest0-21swapper/019:54:000
245033299200,19cyclictest0-21swapper/000:23:240
245033299200,19cyclictest0-21swapper/000:12:360
245033299200,18cyclictest0-21swapper/019:55:220
139099200,14rtkit-daemon0-21swapper/223:47:112
139099200,14rtkit-daemon0-21swapper/222:17:452
2450343991917,1cyclictest3056-21PK-Backend23:44:453
2450343991917,1cyclictest0-21swapper/319:33:413
2450343991911,6cyclictest2550809-21which23:15:153
2450343991911,6cyclictest2505931-21cat21:10:173
245034399190,1cyclictest0-21swapper/323:00:243
245034399190,1cyclictest0-21swapper/321:58:123
2450341991917,1cyclictest0-21swapper/223:08:072
2450341991917,1cyclictest0-21swapper/220:51:122
2450341991916,2cyclictest331rcuc/222:00:082
2450341991916,2cyclictest331rcuc/219:35:092
2450341991916,2cyclictest331rcuc/219:32:232
2450341991915,3cyclictest331rcuc/223:55:122
2450341991915,3cyclictest331rcuc/200:31:452
2450341991915,3cyclictest331rcuc/200:02:102
2450341991913,5cyclictest2518865-21sed21:45:212
2450341991912,5cyclictest2488243-21grep20:20:182
245034199190,1cyclictest0-21swapper/219:40:222
2450337991917,1cyclictest3056-21PK-Backend23:25:021
2450337991917,1cyclictest3056-21PK-Backend00:00:241
2450337991916,2cyclictest3056-21PK-Backend23:14:271
2450337991912,5cyclictest2479349-21cut19:55:171
245033799190,1cyclictest3056-21PK-Backend23:21:581
245033799190,1cyclictest3056-21PK-Backend22:13:371
245033799190,1cyclictest3056-21PK-Backend20:47:591
245033799190,1cyclictest3056-21PK-Backend00:35:591
245033799190,18cyclictest3056-21PK-Backend21:18:481
2450332991917,1cyclictest0-21swapper/022:32:310
2450332991917,1cyclictest0-21swapper/022:20:490
2450332991917,1cyclictest0-21swapper/021:21:130
2450332991917,1cyclictest0-21swapper/021:14:520
2450332991917,1cyclictest0-21swapper/021:08:160
2450332991917,1cyclictest0-21swapper/020:51:260
2450332991917,1cyclictest0-21swapper/020:45:510
2450332991917,1cyclictest0-21swapper/020:43:220
2450332991917,1cyclictest0-21swapper/020:07:590
2450332991912,5cyclictest2502554-21sed21:00:160
2450332991911,6cyclictest2469051-21ntpq19:25:210
2450332991910,7cyclictest2545687-21ntp_states23:00:190
245033299190,1cyclictest0-21swapper/023:55:590
245033299190,1cyclictest0-21swapper/022:44:550
245033299190,1cyclictest0-21swapper/022:17:170
245033299190,1cyclictest0-21swapper/021:50:410
245033299190,1cyclictest0-21swapper/019:47:380
245033299190,18cyclictest0-21swapper/023:50:180
245033299190,18cyclictest0-21swapper/023:30:580
245033299190,18cyclictest0-21swapper/023:29:570
245033299190,18cyclictest0-21swapper/021:46:350
245033299190,18cyclictest0-21swapper/020:55:520
245033299190,18cyclictest0-21swapper/019:41:160
245033299190,18cyclictest0-21swapper/000:17:290
245033299190,18cyclictest0-21swapper/000:08:440
245033299190,18cyclictest0-21swapper/000:08:440
245033299190,18cyclictest0-21swapper/000:03:160
24667832180,3chrt0-21swapper/119:20:131
2450343991816,1cyclictest0-21swapper/321:09:533
245034399181,15cyclictest2527893-21users22:10:223
2450343991811,5cyclictest2507339-21sort21:15:153
2450337991816,1cyclictest3056-21PK-Backend23:32:461
2450337991816,1cyclictest3056-21PK-Backend21:45:591
2450337991816,1cyclictest3056-21PK-Backend20:33:261
2450332991813,3cyclictest2484245-21grep20:10:120
2450332991811,5cyclictest2558866-21readlink23:40:010
2450332991811,5cyclictest2539965-21awk22:45:160
2450332991811,5cyclictest2511235-21cat21:25:180
245033299180,1cyclictest0-21swapper/023:11:320
2450343991715,1cyclictest0-21swapper/320:25:523
245034399170,3cyclictest2528936-21diskstats22:15:153
245034399170,3cyclictest2467423-21python319:20:203
245033799179,6cyclictest2561432-21df23:45:131
2450337991715,1cyclictest3056-21PK-Backend22:48:391
2450337991714,2cyclictest3056-21PK-Backend20:41:511
245033299178,7cyclictest2237-21gdbus21:44:520
2450332991711,4cyclictest2489504-21cpuspeed_turbos20:25:130
2450332991710,5cyclictest2486077-21expr20:15:120
2450332991710,5cyclictest2486077-21expr20:15:110
2450332991710,5cyclictest13-21ksoftirqd/000:25:190
245033799169,5cyclictest26-21ksoftirqd/122:30:211
2450337991610,4cyclictest2470048-21cpu19:30:121
245033299169,5cyclictest2514879-21ntpq21:35:180
2450332991612,3cyclictest2526538-21cron22:10:000
2450332991610,4cyclictest2493664-21grep20:35:210
2450332991610,4cyclictest1095-21runrttasks23:15:060
245033299160,1cyclictest0-21swapper/000:31:080
2450343991510,4cyclictest3056-21PK-Backend19:56:163
2450337991513,1cyclictest3056-21PK-Backend22:20:541
2450337991513,1cyclictest3056-21PK-Backend21:44:371
245033299149,3cyclictest2547685-21timerwakeupswit23:05:230
245033299149,3cyclictest2523740-21tail22:00:170
2450337991311,1cyclictest3056-21PK-Backend23:17:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional