You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 11:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Mon Apr 27, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
132519922270,10sleep342-21ksoftirqd/319:05:113
142538021120,4sleep013-21ksoftirqd/023:50:000
139099870,52rtkit-daemon0-21swapper/019:05:390
132587827139,25sleep10-21swapper/119:05:231
132630726646,14sleep20-21swapper/219:05:402
14182032530,2sleep10-21swapper/123:30:001
13769502490,8sleep234-21ksoftirqd/221:30:162
13762792430,2sleep20-21swapper/221:29:552
14247592350,3sleep10-21swapper/123:45:201
1326770992723,3cyclictest2164-21gdbus21:54:190
139099240,16rtkit-daemon3056-21PK-Backend21:17:080
139099240,16rtkit-daemon3056-21PK-Backend00:18:481
132677699240,23cyclictest1386956-21cron22:04:552
139099230,16rtkit-daemon3056-21PK-Backend22:24:191
139099230,16rtkit-daemon3056-21PK-Backend19:57:431
139099230,16rtkit-daemon3056-21PK-Backend00:13:501
139099230,15rtkit-daemon3056-21PK-Backend21:27:141
139099230,15rtkit-daemon3056-21PK-Backend20:59:031
139099230,15rtkit-daemon3056-21PK-Backend20:53:011
139099230,14rtkit-daemon0-21swapper/019:11:160
1326780992317,5cyclictest883-21dbus-daemon19:28:033
1326780992316,5cyclictest1344560-21apt-get20:00:003
1326776992310,11cyclictest1371132-21unin-run21:15:012
132677699230,3cyclictest1422650-21ntp_states23:40:202
132677499230,21cyclictest1419181-21ntp_states23:30:181
132677499230,21cyclictest1419181-21ntp_states23:30:181
139099220,15rtkit-daemon3056-21PK-Backend23:10:581
139099220,14rtkit-daemon3056-21PK-Backend23:06:511
139099220,14rtkit-daemon3056-21PK-Backend21:51:571
139099220,14rtkit-daemon3056-21PK-Backend19:18:051
139099220,13rtkit-daemon3056-21PK-Backend19:27:361
13672462220,4sleep13056-21PK-Backend21:00:231
132678099226,4cyclictest1411590-21sort23:10:133
1326780992216,4cyclictest1389031-21cat22:05:133
132677699220,20cyclictest1424269-21df_inode23:45:132
1326774992219,2cyclictest3056-21PK-Backend20:13:211
132677499220,21cyclictest3056-21PK-Backend22:52:261
132677499220,20cyclictest3056-21PK-Backend20:44:171
132677499220,20cyclictest3056-21PK-Backend19:22:441
1326770992220,1cyclictest0-21swapper/023:13:340
1326770992220,1cyclictest0-21swapper/023:09:420
1326770992220,1cyclictest0-21swapper/021:41:200
1326770992220,1cyclictest0-21swapper/020:05:280
1326770992220,1cyclictest0-21swapper/019:31:560
1326770992220,1cyclictest0-21swapper/019:31:560
1326770992220,1cyclictest0-21swapper/000:07:140
14108192210,4sleep2331rcuc/223:05:232
139099210,15rtkit-daemon3056-21PK-Backend19:32:421
139099210,15rtkit-daemon3056-21PK-Backend19:32:411
139099210,15rtkit-daemon0-21swapper/100:06:461
139099210,15rtkit-daemon0-21swapper/022:25:540
139099210,14rtkit-daemon3056-21PK-Backend22:01:531
139099210,14rtkit-daemon3056-21PK-Backend19:46:401
139099210,13rtkit-daemon3056-21PK-Backend20:19:151
132678099218,11cyclictest1330931-21/usr/sbin/munin19:20:123
1326780992119,1cyclictest3056-21PK-Backend22:25:273
132678099211,8cyclictest1341281-21latency_hist19:50:003
1326780992115,5cyclictest0-21swapper/321:34:243
1326780992113,6cyclictest1421959-21awk23:40:123
1326780992111,8cyclictest1337651-21unin-run19:40:013
1326780992111,8cyclictest1337651-21unin-run19:40:013
132678099210,8cyclictest1350063-21unin-run20:15:003
132678099210,20cyclictest42-21ksoftirqd/321:44:443
132678099210,20cyclictest3056-21PK-Backend23:15:503
132678099210,19cyclictest3056-21PK-Backend23:36:453
132678099210,19cyclictest3056-21PK-Backend22:31:563
132678099210,19cyclictest3056-21PK-Backend20:32:403
1326776992119,1cyclictest0-21swapper/221:46:162
1326776992113,6cyclictest883-21dbus-daemon19:44:172
1326776992113,6cyclictest1408616-21memory23:00:162
1326776992113,6cyclictest1352403-21df_inode20:20:142
1326776992113,6cyclictest1329222-21apt19:15:122
1326776992112,7cyclictest1384132-21df_inode21:50:152
132677699210,20cyclictest0-21swapper/220:06:192
132677699210,19cyclictest3056-21PK-Backend20:25:322
1326774992119,1cyclictest3056-21PK-Backend22:58:411
1326774992119,1cyclictest3056-21PK-Backend19:37:521
1326774992119,1cyclictest3056-21PK-Backend19:37:521
1326774992119,1cyclictest0-21swapper/122:31:101
1326774992119,1cyclictest0-21swapper/120:25:441
1326774992119,1cyclictest0-21swapper/119:13:181
1326774992119,1cyclictest0-21swapper/100:39:401
1326774992113,6cyclictest1379766-21apt-get21:40:001
1326774992112,7cyclictest1401468-21df22:40:141
132677499210,20cyclictest3056-21PK-Backend21:08:031
132677499210,20cyclictest0-21swapper/123:58:301
132677499210,20cyclictest0-21swapper/122:46:121
132677499210,20cyclictest0-21swapper/122:35:501
132677499210,19cyclictest3056-21PK-Backend23:52:011
132677499210,19cyclictest3056-21PK-Backend21:30:131
1326770992119,1cyclictest0-21swapper/023:59:300
1326770992119,1cyclictest0-21swapper/022:49:480
1326770992119,1cyclictest0-21swapper/022:17:090
1326770992119,1cyclictest0-21swapper/021:11:320
1326770992119,1cyclictest0-21swapper/020:55:590
1326770992119,1cyclictest0-21swapper/020:35:090
1326770992119,1cyclictest0-21swapper/019:55:560
1326770992119,1cyclictest0-21swapper/019:29:050
1326770992119,1cyclictest0-21swapper/000:13:570
1326770992113,6cyclictest436-20systemd-journal22:15:000
132677099210,20cyclictest0-21swapper/023:25:570
132677099210,20cyclictest0-21swapper/023:24:560
132677099210,20cyclictest0-21swapper/021:28:100
132677099210,20cyclictest0-21swapper/019:43:180
1326780992018,1cyclictest3056-21PK-Backend21:45:463
1326780992018,1cyclictest0-21swapper/319:16:013
1326780992013,5cyclictest1411225-21cron23:10:003
1326780992012,6cyclictest1351341-21ls20:15:223
132678099201,16cyclictest1379164-21python321:35:203
132678099201,16cyclictest1367879-21/usr/sbin/munin21:05:133
132678099200,1cyclictest0-21swapper/321:50:263
132678099200,1cyclictest0-21swapper/300:30:053
132678099200,1cyclictest0-21swapper/300:16:263
132678099200,19cyclictest0-21swapper/323:47:203
132678099200,19cyclictest0-21swapper/320:53:273
132678099200,19cyclictest0-21swapper/300:11:333
132678099200,18cyclictest3056-21PK-Backend22:13:173
132678099200,18cyclictest1386222-21ntpq21:55:183
1326776992018,1cyclictest0-21swapper/223:20:462
1326776992018,1cyclictest0-21swapper/200:33:052
1326776992018,1cyclictest0-21swapper/200:28:122
1326776992014,5cyclictest34-21ksoftirqd/221:35:112
1326776992012,6cyclictest1426361-21ntpq23:50:182
1326776992012,6cyclictest1335876-21cron19:34:592
1326776992012,6cyclictest1335876-21cron19:34:592
132677699200,1cyclictest0-21swapper/223:34:252
132677699200,1cyclictest0-21swapper/223:34:252
132677699200,1cyclictest0-21swapper/222:46:462
132677699200,1cyclictest0-21swapper/220:53:172
132677699200,1cyclictest0-21swapper/219:13:262
132677699200,19cyclictest0-21swapper/223:58:522
132677699200,19cyclictest0-21swapper/223:18:432
132677699200,19cyclictest0-21swapper/222:55:312
132677699200,19cyclictest0-21swapper/222:42:542
132677699200,19cyclictest0-21swapper/222:39:022
132677699200,19cyclictest0-21swapper/221:05:272
132677699200,19cyclictest0-21swapper/221:00:472
132677699200,19cyclictest0-21swapper/220:56:072
132677699200,19cyclictest0-21swapper/220:36:412
132677699200,19cyclictest0-21swapper/220:17:212
132677699200,19cyclictest0-21swapper/219:47:332
132677699200,19cyclictest0-21swapper/219:28:132
132677699200,19cyclictest0-21swapper/219:23:342
132677699200,19cyclictest0-21swapper/200:03:452
1326774992018,1cyclictest3056-21PK-Backend23:02:201
1326774992018,1cyclictest3056-21PK-Backend20:39:031
1326774992018,1cyclictest3056-21PK-Backend00:28:181
1326774992018,1cyclictest3056-21PK-Backend00:21:011
1326774992018,1cyclictest0-21swapper/121:47:381
1326774992018,1cyclictest0-21swapper/120:00:571
1326774992018,1cyclictest0-21swapper/100:33:591
1326774992017,2cyclictest3056-21PK-Backend21:57:311
132677499200,1cyclictest0-21swapper/120:31:121
132677499200,19cyclictest3056-21PK-Backend19:54:291
132677499200,19cyclictest0-21swapper/122:12:581
1326770992018,1cyclictest3056-21PK-Backend00:25:320
1326770992018,1cyclictest0-21swapper/023:04:000
1326770992018,1cyclictest0-21swapper/022:35:480
1326770992018,1cyclictest0-21swapper/022:34:470
1326770992018,1cyclictest0-21swapper/022:08:040
1326770992018,1cyclictest0-21swapper/022:04:250
1326770992018,1cyclictest0-21swapper/021:58:100
1326770992018,1cyclictest0-21swapper/020:30:430
1326770992018,1cyclictest0-21swapper/020:22:110
1326770992018,1cyclictest0-21swapper/000:21:410
1326770992018,1cyclictest0-21swapper/000:17:490
1326770992018,1cyclictest0-21swapper/000:03:220
1326770992013,5cyclictest1441681-21tr00:35:130
1326770992013,5cyclictest1-21systemd19:25:000
132677099200,1cyclictest0-21swapper/022:42:170
132677099200,1cyclictest0-21swapper/020:19:070
132677099200,19cyclictest0-21swapper/023:52:470
132677099200,19cyclictest0-21swapper/023:34:290
132677099200,19cyclictest0-21swapper/023:34:290
132677099200,19cyclictest0-21swapper/021:01:250
132677099200,19cyclictest0-21swapper/019:47:100
132677099200,19cyclictest0-21swapper/019:39:270
132677099200,19cyclictest0-21swapper/019:39:260
132677099200,19cyclictest0-21swapper/000:34:040
1326780991917,1cyclictest1211-21kerneloops22:48:463
1326780991917,1cyclictest0-21swapper/322:01:343
1326780991917,1cyclictest0-21swapper/321:20:243
1326780991917,1cyclictest0-21swapper/320:39:553
1326780991916,2cyclictest3056-21PK-Backend23:31:043
1326780991916,2cyclictest3056-21PK-Backend23:31:043
1326780991916,2cyclictest3056-21PK-Backend00:07:413
1326780991914,3cyclictest1334799-21diskstats19:30:153
1326780991914,3cyclictest1334799-21diskstats19:30:153
1326780991911,6cyclictest1360300-21awk20:40:223
132678099191,15cyclictest1408398-21df23:00:143
132678099190,7cyclictest1361696-21ntp_states20:45:183
132678099190,6cyclictest1328490-21systemd-run19:15:013
132678099190,1cyclictest3056-21PK-Backend20:02:323
132678099190,1cyclictest3056-21PK-Backend00:00:583
132678099190,1cyclictest0-21swapper/323:52:133
132678099190,1cyclictest0-21swapper/320:25:233
132678099190,1cyclictest0-21swapper/319:42:093
132678099190,1cyclictest0-21swapper/300:29:043
132678099190,1cyclictest0-21swapper/300:21:203
132678099190,18cyclictest0-21swapper/322:54:273
132678099190,18cyclictest0-21swapper/321:15:453
132678099190,18cyclictest0-21swapper/321:12:543
1326776991917,1cyclictest0-21swapper/223:10:592
1326776991917,1cyclictest0-21swapper/222:51:392
1326776991917,1cyclictest0-21swapper/220:43:092
1326776991917,1cyclictest0-21swapper/219:55:312
1326776991917,1cyclictest0-21swapper/200:18:252
1326776991913,5cyclictest1437425-21sshd00:20:452
1326776991912,5cyclictest1394066-21ls22:20:012
1326776991912,5cyclictest1371521-21expr21:15:132
132677699190,1cyclictest0-21swapper/222:32:192
132677699190,18cyclictest0-21swapper/223:25:392
132677699190,18cyclictest0-21swapper/222:05:572
132677699190,18cyclictest0-21swapper/220:14:172
132677699190,18cyclictest0-21swapper/219:53:282
1326774991917,1cyclictest0-21swapper/123:35:111
1326774991916,2cyclictest3056-21PK-Backend22:07:451
1326774991916,2cyclictest3056-21PK-Backend21:12:291
1326774991914,3cyclictest1413883-21cut23:15:181
1326774991912,5cyclictest1347583-21open_files20:05:191
132677499190,1cyclictest0-21swapper/120:48:441
132677499190,18cyclictest0-21swapper/121:40:071
132677499190,18cyclictest0-21swapper/121:16:561
1326770991917,1cyclictest0-21swapper/023:40:240
1326770991917,1cyclictest0-21swapper/023:17:260
1326770991917,1cyclictest0-21swapper/022:56:310
1326770991917,1cyclictest0-21swapper/021:38:300
1326770991917,1cyclictest0-21swapper/020:28:550
1326770991917,1cyclictest0-21swapper/020:04:270
1326770991917,1cyclictest0-21swapper/019:54:540
132677099190,1cyclictest0-21swapper/020:41:240
132677099190,18cyclictest0-21swapper/021:30:590
132677099190,18cyclictest0-21swapper/021:08:420
132677099190,18cyclictest0-21swapper/020:46:520
132678099185,11cyclictest1364413-21/usr/sbin/munin20:55:163
1326780991816,1cyclictest3056-21PK-Backend20:07:243
1326780991816,1cyclictest0-21swapper/323:25:243
1326780991816,1cyclictest0-21swapper/322:58:193
1326780991816,1cyclictest0-21swapper/320:22:183
1326780991813,3cyclictest883-21dbus-daemon00:00:003
1326780991813,3cyclictest1366652-21kernelversion21:00:183
1326780991812,5cyclictest1394733-21cat22:20:173
132678099180,1cyclictest0-21swapper/321:26:533
1326776991816,1cyclictest0-21swapper/222:20:572
1326776991816,1cyclictest0-21swapper/222:10:362
1326776991816,1cyclictest0-21swapper/220:46:002
1326776991816,1cyclictest0-21swapper/200:10:412
1326776991812,5cyclictest1345087-21apt20:00:102
1326776991811,5cyclictest1386071-21cat21:55:172
1326774991816,1cyclictest3056-21PK-Backend22:17:381
132677499180,1cyclictest3056-21PK-Backend19:41:301
132677499180,17cyclictest3056-21PK-Backend23:40:531
1326770991816,1cyclictest0-21swapper/023:36:320
1326770991816,1cyclictest0-21swapper/021:47:010
1326770991816,1cyclictest0-21swapper/020:10:220
1326770991816,1cyclictest0-21swapper/019:19:590
1326780991715,1cyclictest3056-21PK-Backend00:36:483
1326780991715,1cyclictest0-21swapper/322:19:463
132678099170,15cyclictest1341742-21diskstats19:50:153
1326776991711,4cyclictest1380869-21load21:40:162
1326776991710,5cyclictest1336489-21expr19:35:132
1326776991710,5cyclictest1336489-21expr19:35:132
1326774991715,1cyclictest3056-21PK-Backend20:24:431
1326774991715,1cyclictest0-21swapper/123:21:461
1326774991712,3cyclictest1397360-21cron22:29:591
1326770991715,1cyclictest0-21swapper/022:21:490
1326770991715,1cyclictest0-21swapper/020:52:190
1326770991711,4cyclictest1405440-21smartctl22:50:220
1326780991614,1cyclictest3056-21PK-Backend23:24:213
1326776991614,1cyclictest0-21swapper/223:37:292
1326776991614,1cyclictest0-21swapper/222:29:292
1326776991614,1cyclictest0-21swapper/200:05:482
1326776991612,3cyclictest1373105-21apt21:20:122
1326774991614,1cyclictest0-21swapper/100:02:221
132677699154,9cyclictest1356051-21latency20:30:172
1326776991513,1cyclictest0-21swapper/200:36:102
1326774991513,1cyclictest3056-21PK-Backend21:23:101
1326770991513,1cyclictest0-21swapper/021:20:520
1326780991411,2cyclictest3056-21PK-Backend22:35:483
1326780991410,3cyclictest1401083-21/usr/sbin/munin22:40:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional