You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-05 - 04:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Apr 05, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
143420221300,4sleep23056-21PK-Backend20:45:012
142153621290,3sleep20-21swapper/220:05:232
141192321290,4sleep20-21swapper/219:40:132
143863221100,4sleep33056-21PK-Backend20:55:173
139996327639,13sleep30-21swapper/319:08:063
139099750,57rtkit-daemon0-21swapper/019:05:130
139952827439,9sleep20-21swapper/219:05:252
140005826142,13sleep10-21swapper/119:09:211
14541432560,2sleep33056-21PK-Backend21:40:113
15014652470,5sleep33056-21PK-Backend23:55:003
14694372400,4sleep11469419-21perf22:25:001
14448452380,4sleep23056-21PK-Backend21:15:002
1400287993313,9cyclictest34-21ksoftirqd/200:40:012
1400287993313,10cyclictest34-21ksoftirqd/200:20:012
1400287993112,10cyclictest34-21ksoftirqd/221:40:012
1400287993013,4cyclictest34-21ksoftirqd/222:05:012
1400287993013,4cyclictest34-21ksoftirqd/222:05:012
1400287993013,4cyclictest34-21ksoftirqd/200:30:012
1400287993010,10cyclictest34-21ksoftirqd/219:59:592
140028999261,1cyclictest141rcu_preempt23:40:013
140028799268,9cyclictest34-21ksoftirqd/200:00:002
140028799267,10cyclictest34-21ksoftirqd/219:30:002
140028799260,6cyclictest34-21ksoftirqd/222:45:132
1400284992612,12cyclictest1407914-21unin-run19:30:001
1400280992615,8cyclictest436-20systemd-journal20:45:000
140028999255,11cyclictest141rcu_preempt20:45:003
140028799256,6cyclictest34-21ksoftirqd/220:20:012
140028799256,10cyclictest34-21ksoftirqd/219:35:002
140028799253,7cyclictest2162-21gmain00:05:002
140028799250,5cyclictest1480063-21cron22:54:592
14646742240,4sleep03056-21PK-Backend22:10:110
14328022240,3sleep126-21ksoftirqd/120:40:101
1400289992413,9cyclictest978-21snmpd21:55:253
140028799246,7cyclictest34-21ksoftirqd/223:05:002
1400287992420,3cyclictest331rcuc/220:59:422
1400284992415,7cyclictest1426133-21df20:20:131
140028099240,23cyclictest1389-21rtkit-daemon20:14:070
139099240,4rtkit-daemon1389-21rtkit-daemon19:55:330
140028999230,2cyclictest141rcu_preempt20:50:003
140028799237,4cyclictest34-21ksoftirqd/223:40:002
140028799235,9cyclictest34-21ksoftirqd/221:35:012
140028799234,6cyclictest34-21ksoftirqd/200:05:132
140028799233,6cyclictest34-21ksoftirqd/223:10:122
140028799232,7cyclictest34-21ksoftirqd/220:50:202
1400287992319,3cyclictest331rcuc/221:54:122
1400287992319,3cyclictest331rcuc/219:11:412
1400287992315,6cyclictest1476294-21sshd22:41:102
140028799230,8cyclictest1510612-21cron00:20:012
140028999221,19cyclictest1403675-21ntp_states19:15:183
140028999220,20cyclictest1464076-21sshd22:09:093
140028799227,4cyclictest34-21ksoftirqd/222:35:322
140028799225,9cyclictest34-21ksoftirqd/220:00:182
140028799223,7cyclictest34-21ksoftirqd/223:40:162
140028799223,6cyclictest34-21ksoftirqd/223:05:012
140028799222,6cyclictest3056-21PK-Backend23:15:222
1400287992219,2cyclictest331rcuc/221:06:422
1400287992218,3cyclictest331rcuc/223:48:042
1400287992218,3cyclictest331rcuc/222:31:352
1400287992218,3cyclictest331rcuc/221:46:222
1400287992218,3cyclictest331rcuc/221:15:112
140028799221,7cyclictest34-21ksoftirqd/221:20:162
140028799220,6cyclictest1416552-21munin-run19:54:592
1400284992220,1cyclictest3056-21PK-Backend19:36:551
1400284992220,1cyclictest0-21swapper/123:02:171
1400284992220,1cyclictest0-21swapper/121:59:101
1400284992220,1cyclictest0-21swapper/121:23:001
1400284992220,1cyclictest0-21swapper/121:15:311
1400284992220,1cyclictest0-21swapper/121:10:041
1400284992220,1cyclictest0-21swapper/119:44:321
1400284992220,1cyclictest0-21swapper/119:21:401
1400284992220,1cyclictest0-21swapper/100:18:561
1400284992220,1cyclictest0-21swapper/100:07:201
1400284992219,2cyclictest3056-21PK-Backend23:32:531
1400280992217,3cyclictest1405420-21ntp_states19:20:180
140028099220,21cyclictest0-21swapper/022:46:480
139099220,4rtkit-daemon1389-21rtkit-daemon23:11:571
139099220,18rtkit-daemon3056-21PK-Backend21:06:061
140028999215,14cyclictest1401162-21ntp_kernel_pll_19:10:173
140028999212,2cyclictest141rcu_preempt20:00:003
1400289992118,2cyclictest3056-21PK-Backend20:07:503
1400289992118,2cyclictest3056-21PK-Backend00:09:413
1400289992116,2cyclictest42-21ksoftirqd/321:15:003
1400289992114,5cyclictest1467308-21sshd22:16:093
140028999211,18cyclictest1479472-21python322:50:213
140028999210,20cyclictest3056-21PK-Backend23:16:163
140028999210,20cyclictest3056-21PK-Backend21:51:533
140028999210,20cyclictest0-21swapper/323:13:103
140028999210,19cyclictest3056-21PK-Backend22:45:393
140028999210,19cyclictest3056-21PK-Backend20:11:273
140028799216,7cyclictest3056-21PK-Backend22:20:012
140028799215,5cyclictest34-21ksoftirqd/220:35:172
140028799213,5cyclictest34-21ksoftirqd/221:25:162
140028799212,5cyclictest34-21ksoftirqd/221:45:012
1400287992118,2cyclictest331rcuc/222:07:042
1400287992118,2cyclictest331rcuc/221:55:292
1400287992118,2cyclictest331rcuc/200:32:432
1400287992117,3cyclictest331rcuc/222:15:222
1400287992117,3cyclictest331rcuc/222:10:522
140028799210,7cyclictest34-21ksoftirqd/223:25:122
140028799210,6cyclictest34-21ksoftirqd/219:35:132
140028799210,20cyclictest0-21swapper/219:15:372
1400284992119,1cyclictest3056-21PK-Backend00:14:021
1400284992119,1cyclictest0-21swapper/123:52:261
1400284992119,1cyclictest0-21swapper/123:39:481
1400284992119,1cyclictest0-21swapper/122:48:251
1400284992119,1cyclictest0-21swapper/122:35:471
1400284992119,1cyclictest0-21swapper/122:03:021
1400284992119,1cyclictest0-21swapper/122:03:021
1400284992119,1cyclictest0-21swapper/121:32:341
1400284992119,1cyclictest0-21swapper/121:01:101
1400284992118,2cyclictest1436837-21smartctl20:50:161
1400284992115,4cyclictest1472870-21cron22:35:001
140028499210,20cyclictest0-21swapper/122:28:511
140028499210,20cyclictest0-21swapper/121:44:581
1400280992113,6cyclictest2399-21gdbus22:28:030
1400280992112,7cyclictest1508630-21apt-get00:15:010
140028099210,20cyclictest3056-21PK-Backend19:15:170
140028099210,20cyclictest0-21swapper/019:45:110
139099210,4rtkit-daemon953-21gmain22:55:130
139099210,4rtkit-daemon953-21gmain22:55:130
139099210,3rtkit-daemon1389-21rtkit-daemon21:10:060
140028999207,2cyclictest141rcu_preempt21:16:163
140028999202,11cyclictest141rcu_preempt00:00:213
140028999201,9cyclictest42-21ksoftirqd/322:00:213
140028999201,9cyclictest42-21ksoftirqd/322:00:213
1400289992018,1cyclictest3056-21PK-Backend23:41:573
1400289992017,2cyclictest3056-21PK-Backend22:38:433
1400289992015,4cyclictest42-21ksoftirqd/300:10:443
140028999200,1cyclictest3056-21PK-Backend20:36:363
140028999200,19cyclictest3056-21PK-Backend23:23:113
140028999200,19cyclictest3056-21PK-Backend23:05:133
140028999200,19cyclictest3056-21PK-Backend22:55:273
140028999200,19cyclictest3056-21PK-Backend22:55:263
140028999200,19cyclictest3056-21PK-Backend22:25:533
140028999200,19cyclictest3056-21PK-Backend22:22:003
140028999200,19cyclictest3056-21PK-Backend20:31:563
140028999200,19cyclictest3056-21PK-Backend19:27:073
140028999200,19cyclictest3056-21PK-Backend00:34:223
140028999200,18cyclictest3056-21PK-Backend19:45:593
1400287992017,2cyclictest331rcuc/223:30:482
1400287992016,2cyclictest331rcuc/223:20:362
1400287992015,3cyclictest978-21snmpd23:51:492
1400287992015,3cyclictest34-21ksoftirqd/219:50:012
140028799200,7cyclictest34-21ksoftirqd/220:50:002
140028799200,6cyclictest1423778-21munin-run20:15:012
1400284992019,1cyclictest0-21swapper/100:30:331
1400284992018,1cyclictest3056-21PK-Backend19:58:041
1400284992018,1cyclictest3056-21PK-Backend19:14:321
1400284992018,1cyclictest0-21swapper/123:41:511
1400284992018,1cyclictest0-21swapper/123:05:221
1400284992018,1cyclictest0-21swapper/122:50:271
1400284992018,1cyclictest0-21swapper/121:26:521
1400284992018,1cyclictest0-21swapper/120:06:081
1400284992018,1cyclictest0-21swapper/100:37:141
1400284992017,2cyclictest3056-21PK-Backend00:29:301
140028499200,1cyclictest0-21swapper/123:20:011
140028499200,19cyclictest3056-21PK-Backend20:03:181
140028499200,19cyclictest0-21swapper/123:58:211
140028499200,19cyclictest0-21swapper/122:08:441
140028499200,19cyclictest0-21swapper/120:27:581
140028499200,19cyclictest0-21swapper/100:00:241
140028099209,9cyclictest1409645-21unin-run19:35:000
1400280992018,1cyclictest0-21swapper/023:34:470
1400280992018,1cyclictest0-21swapper/021:44:010
1400280992013,5cyclictest1438144-21/usr/sbin/munin20:55:120
1400280992012,6cyclictest1447856-21awk21:20:190
1400280992012,6cyclictest1437015-21memory20:50:170
1400280992010,3cyclictest1389-21rtkit-daemon21:33:410
140028099200,1cyclictest0-21swapper/021:54:370
140028099200,1cyclictest0-21swapper/020:36:150
140028099200,19cyclictest3056-21PK-Backend19:35:320
140028099200,19cyclictest3056-21PK-Backend00:30:300
140028099200,19cyclictest0-21swapper/023:24:470
140028099200,19cyclictest0-21swapper/022:37:490
140028099200,19cyclictest0-21swapper/020:46:220
139099200,3rtkit-daemon1389-21rtkit-daemon00:06:280
140028999199,4cyclictest42-21ksoftirqd/300:15:033
140028999194,2cyclictest141rcu_preempt20:15:203
1400289991917,1cyclictest3056-21PK-Backend23:29:063
1400289991917,1cyclictest3056-21PK-Backend21:38:273
1400289991917,1cyclictest0-21swapper/323:47:513
1400289991917,1cyclictest0-21swapper/320:24:253
1400289991917,1cyclictest0-21swapper/319:30:323
1400289991917,1cyclictest0-21swapper/300:25:373
140028999191,5cyclictest1441723-21sort21:05:123
1400289991912,6cyclictest1456809-21sed21:45:213
140028999191,16cyclictest1410356-21irqstats19:35:153
140028999190,2cyclictest3056-21PK-Backend23:55:143
140028999190,2cyclictest3056-21PK-Backend23:03:103
140028999190,1cyclictest3056-21PK-Backend21:34:353
140028999190,1cyclictest3056-21PK-Backend20:53:593
140028999190,1cyclictest3056-21PK-Backend19:21:183
140028999190,1cyclictest0-21swapper/322:43:363
140028999190,18cyclictest3056-21PK-Backend19:52:143
140028999190,18cyclictest3056-21PK-Backend00:23:343
140028999190,18cyclictest0-21swapper/322:32:483
140028999190,18cyclictest0-21swapper/321:03:193
140028999190,18cyclictest0-21swapper/319:40:453
140028999190,18cyclictest0-21swapper/300:35:233
140028799196,4cyclictest34-21ksoftirqd/222:25:202
140028799196,3cyclictest34-21ksoftirqd/222:55:212
140028799196,3cyclictest34-21ksoftirqd/222:55:202
1400287991916,2cyclictest331rcuc/200:10:102
140028799190,7cyclictest34-21ksoftirqd/220:26:352
1400284991917,1cyclictest0-21swapper/123:27:591
1400284991917,1cyclictest0-21swapper/122:56:221
1400284991917,1cyclictest0-21swapper/122:56:221
1400284991917,1cyclictest0-21swapper/120:55:431
1400284991911,6cyclictest1429534-21cat20:30:131
140028499190,4cyclictest0-21swapper/122:40:041
140028499190,1cyclictest3056-21PK-Backend19:16:551
140028499190,1cyclictest0-21swapper/121:35:251
140028499190,1cyclictest0-21swapper/120:38:191
140028499190,1cyclictest0-21swapper/120:19:541
140028499190,18cyclictest3056-21PK-Backend23:46:451
140028099193,15cyclictest1095-21runrttasks23:19:250
1400280991917,1cyclictest3056-21PK-Backend22:40:530
1400280991917,1cyclictest0-21swapper/023:45:490
1400280991917,1cyclictest0-21swapper/023:27:510
1400280991917,1cyclictest0-21swapper/022:30:530
1400280991917,1cyclictest0-21swapper/021:59:290
1400280991917,1cyclictest0-21swapper/021:17:110
1400280991917,1cyclictest0-21swapper/021:08:400
1400280991917,1cyclictest0-21swapper/020:23:040
1400280991917,1cyclictest0-21swapper/000:17:390
1400280991917,1cyclictest0-21swapper/000:03:470
1400280991912,6cyclictest1430155-21ls20:30:230
1400280991912,5cyclictest1453072-21pluginstate21:35:190
1400280991911,6cyclictest1456146-21cat21:45:150
140028099190,1cyclictest0-21swapper/023:53:460
140028099190,1cyclictest0-21swapper/023:43:470
140028099190,1cyclictest0-21swapper/022:21:550
140028099190,1cyclictest0-21swapper/020:09:520
140028099190,1cyclictest0-21swapper/019:50:390
140028099190,1cyclictest0-21swapper/000:36:250
140028099190,18cyclictest0-21swapper/021:26:450
140028099190,18cyclictest0-21swapper/000:22:330
14277882180,15sleep03056-21PK-Backend20:25:130
140028999183,2cyclictest141rcu_preempt20:05:013
1400289991816,1cyclictest0-21swapper/320:27:163
1400289991811,5cyclictest1465479-21sed22:10:203
140028999180,6cyclictest1493513-21latency23:30:163
140028799183,6cyclictest34-21ksoftirqd/220:20:102
140028799180,6cyclictest34-21ksoftirqd/219:20:192
1400284991816,1cyclictest0-21swapper/122:12:361
1400284991816,1cyclictest0-21swapper/120:10:481
1400284991812,5cyclictest1488023-21expr23:15:111
1400284991812,5cyclictest1409197-21sed19:30:221
1400280991816,1cyclictest0-21swapper/023:02:430
1400280991816,1cyclictest0-21swapper/022:53:440
1400280991811,6cyclictest1502114-21cat23:55:160
1400280991811,5cyclictest1495457-21ntpq23:35:170
1400280991810,6cyclictest1425523-21sleep20:20:000
140028999178,3cyclictest141rcu_preempt21:25:123
1400289991713,2cyclictest42-21ksoftirqd/321:23:273
1400287991714,2cyclictest331rcuc/220:30:302
1400287991710,5cyclictest1440005-21cat21:00:132
140028499178,8cyclictest3056-21PK-Backend21:54:591
1400284991715,1cyclictest3056-21PK-Backend21:48:491
1400284991711,5cyclictest1415184-21cat19:50:131
1400280991715,1cyclictest3056-21PK-Backend23:05:470
1400280991715,1cyclictest0-21swapper/023:13:450
1400280991715,1cyclictest0-21swapper/021:04:000
1400280991712,4cyclictest3056-21PK-Backend22:19:020
1400280991712,3cyclictest1513354-21ntp_offset00:25:180
1400280991710,5cyclictest1419384-21cat20:00:180
139099170,14rtkit-daemon0-21swapper/022:05:130
140028499169,6cyclictest1435404-21ntp_kernel_err20:45:191
1400284991614,1cyclictest0-21swapper/119:48:111
1400284991611,3cyclictest1511338-21if_enp2s000:20:171
140028499160,15cyclictest0-21swapper/122:19:041
140028099165,2cyclictest141rcu_preempt19:25:120
1400280991610,5cyclictest1461191-21cat22:00:130
1400280991610,5cyclictest1461191-21cat22:00:130
1400280991513,1cyclictest0-21swapper/019:10:300
140028099149,3cyclictest1411728-21cpuspeed_turbos19:40:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional