You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-03 - 09:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Jul 03, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
148986521340,3sleep30-21swapper/319:10:143
14920022970,3sleep13056-21PK-Backend19:15:131
139099850,51rtkit-daemon0-21swapper/219:08:572
148772627846,22sleep30-21swapper/319:05:123
148897227639,13sleep00-21swapper/019:07:060
148861327147,15sleep10-21swapper/119:05:251
15591972520,1sleep31559203-21grep22:25:223
15322952490,4sleep33056-21PK-Backend21:10:153
15492572400,3sleep01549251-21apt-config22:00:000
15827942390,3sleep10-21swapper/123:35:011
1489353992817,4cyclictest26-21ksoftirqd/121:20:011
16024242260,3sleep234-21ksoftirqd/200:30:152
15597742260,5sleep10-21swapper/122:30:001
1489353992611,4cyclictest26-21ksoftirqd/123:29:501
148935999251,20cyclictest1544034-21apt21:45:123
148935899250,23cyclictest1095-21runrttasks21:17:482
1489353992417,5cyclictest26-21ksoftirqd/123:55:001
139099240,16rtkit-daemon3056-21PK-Backend22:37:113
1489358992320,2cyclictest0-21swapper/200:04:042
148935399232,19cyclictest1554417-21grep22:15:001
139099230,14rtkit-daemon0-21swapper/200:35:542
15135562220,2sleep33056-21PK-Backend20:15:183
1489359992219,2cyclictest0-21swapper/322:15:123
1489359992216,5cyclictest883-21dbus-daemon21:38:023
148935999220,20cyclictest3056-21PK-Backend20:04:263
1489358992220,1cyclictest0-21swapper/222:33:392
1489358992220,1cyclictest0-21swapper/220:20:162
1489358992220,1cyclictest0-21swapper/219:40:072
1489358992219,2cyclictest0-21swapper/221:44:382
1489358992219,2cyclictest0-21swapper/219:32:442
1489358992217,3cyclictest1536806-21cron21:25:002
148935899220,20cyclictest3056-21PK-Backend20:25:022
1489353992220,1cyclictest0-21swapper/100:35:211
1489353992216,4cyclictest1510812-21sh20:10:001
1489353992215,5cyclictest1586173-21systemctl23:45:001
148935399220,20cyclictest3056-21PK-Backend19:29:271
1489350992211,9cyclictest1523191-21/usr/sbin/munin20:45:220
139099220,16rtkit-daemon3056-21PK-Backend20:51:423
139099220,15rtkit-daemon3056-21PK-Backend21:22:013
139099220,14rtkit-daemon0-21swapper/221:11:112
148935999210,1cyclictest3056-21PK-Backend00:38:493
148935999210,19cyclictest3056-21PK-Backend23:37:053
148935999210,19cyclictest3056-21PK-Backend22:47:573
148935999210,19cyclictest3056-21PK-Backend20:49:143
148935999210,19cyclictest3056-21PK-Backend20:12:373
148935999210,18cyclictest436-20systemd-journal19:30:443
1489358992119,1cyclictest0-21swapper/223:56:062
1489358992119,1cyclictest0-21swapper/223:19:292
1489358992119,1cyclictest0-21swapper/223:08:402
1489358992119,1cyclictest0-21swapper/223:03:412
1489358992119,1cyclictest0-21swapper/222:24:272
1489358992119,1cyclictest0-21swapper/222:16:352
1489358992119,1cyclictest0-21swapper/222:00:202
1489358992119,1cyclictest0-21swapper/221:32:152
1489358992119,1cyclictest0-21swapper/221:08:222
1489358992119,1cyclictest0-21swapper/220:14:552
1489358992119,1cyclictest0-21swapper/200:21:002
1489358992118,2cyclictest0-21swapper/221:39:182
1489358992113,6cyclictest1579315-21sed23:25:012
148935899210,20cyclictest0-21swapper/223:41:322
148935899210,20cyclictest0-21swapper/222:46:092
148935899210,20cyclictest0-21swapper/222:27:312
148935899210,20cyclictest0-21swapper/222:13:312
148935899210,20cyclictest0-21swapper/221:56:422
148935899210,20cyclictest0-21swapper/220:58:012
148935899210,20cyclictest0-21swapper/220:41:522
148935899210,20cyclictest0-21swapper/220:38:482
148935899210,20cyclictest0-21swapper/219:59:072
148935899210,20cyclictest0-21swapper/219:59:072
148935899210,1cyclictest0-21swapper/222:41:102
148935899210,1cyclictest0-21swapper/222:41:092
148935399218,11cyclictest1521249-21systemd20:40:011
1489353992119,1cyclictest3056-21PK-Backend22:17:521
1489353992119,1cyclictest3056-21PK-Backend21:59:201
1489353992119,1cyclictest3056-21PK-Backend19:44:351
1489353992119,1cyclictest3056-21PK-Backend00:14:181
1489353992119,1cyclictest0-21swapper/123:45:111
1489353992119,1cyclictest0-21swapper/123:01:251
1489353992119,1cyclictest0-21swapper/121:21:131
1489353992119,1cyclictest0-21swapper/121:10:061
1489353992119,1cyclictest0-21swapper/100:25:481
1489353992118,2cyclictest3056-21PK-Backend21:53:591
1489353992118,2cyclictest3056-21PK-Backend21:53:581
1489353992118,2cyclictest3056-21PK-Backend19:13:251
1489353992116,4cyclictest26-21ksoftirqd/100:00:011
1489353992115,4cyclictest1518210-21cpuspeed_turbos20:30:131
1489353992113,6cyclictest1600029-21cron00:25:011
1489353992113,6cyclictest1557364-21sed22:20:221
148935399210,20cyclictest3056-21PK-Backend19:36:361
148935399210,20cyclictest0-21swapper/120:47:421
148935399210,19cyclictest3056-21PK-Backend23:24:301
148935099210,20cyclictest0-21swapper/022:55:230
148935099210,19cyclictest3056-21PK-Backend00:05:210
139099210,15rtkit-daemon3056-21PK-Backend20:38:423
148935999209,8cyclictest1788610-21gdbus21:41:493
1489359992018,1cyclictest0-21swapper/319:15:043
1489359992018,1cyclictest0-21swapper/300:13:073
1489359992017,2cyclictest3056-21PK-Backend22:57:093
1489359992017,2cyclictest3056-21PK-Backend21:18:343
1489359992017,2cyclictest3056-21PK-Backend20:40:153
148935999200,1cyclictest3056-21PK-Backend22:32:023
148935999200,1cyclictest0-21swapper/323:20:283
148935999200,19cyclictest3056-21PK-Backend23:48:403
148935999200,19cyclictest3056-21PK-Backend22:06:123
148935999200,19cyclictest3056-21PK-Backend22:00:173
148935999200,19cyclictest3056-21PK-Backend21:09:413
148935999200,19cyclictest3056-21PK-Backend19:57:363
148935999200,19cyclictest3056-21PK-Backend19:57:363
148935999200,19cyclictest0-21swapper/321:56:063
148935999200,19cyclictest0-21swapper/319:45:463
148935999200,19cyclictest0-21swapper/319:40:193
148935999200,19cyclictest0-21swapper/300:21:533
148935999200,19cyclictest0-21swapper/300:17:473
1489358992018,1cyclictest0-21swapper/221:53:032
1489358992018,1cyclictest0-21swapper/221:53:032
1489358992018,1cyclictest0-21swapper/220:53:012
1489358992018,1cyclictest0-21swapper/220:34:362
1489358992018,1cyclictest0-21swapper/220:07:182
1489358992018,1cyclictest0-21swapper/219:49:412
1489358992018,1cyclictest0-21swapper/219:23:042
1489358992013,5cyclictest1581934-21awk23:30:192
1489358992010,8cyclictest1594856-21unin-run00:10:002
148935899200,1cyclictest1895009-21snapd22:39:142
148935899200,19cyclictest0-21swapper/223:51:262
148935899200,19cyclictest0-21swapper/223:25:172
148935899200,19cyclictest0-21swapper/223:10:372
148935899200,19cyclictest0-21swapper/222:05:072
148935899200,19cyclictest0-21swapper/221:04:232
148935899200,19cyclictest0-21swapper/220:48:002
148935899200,19cyclictest0-21swapper/219:54:342
148935899200,19cyclictest0-21swapper/219:54:332
148935899200,19cyclictest0-21swapper/219:27:302
1489353992018,1cyclictest3056-21PK-Backend21:49:131
1489353992018,1cyclictest3056-21PK-Backend21:42:101
1489353992018,1cyclictest0-21swapper/122:53:201
1489353992018,1cyclictest0-21swapper/122:53:201
1489353992018,1cyclictest0-21swapper/122:47:471
1489353992018,1cyclictest0-21swapper/122:39:421
1489353992018,1cyclictest0-21swapper/121:00:461
1489353992018,1cyclictest0-21swapper/120:50:111
1489353992018,1cyclictest0-21swapper/120:17:201
1489353992014,4cyclictest2156-21gnome-shell00:31:091
1489353992013,5cyclictest883-21dbus-daemon20:00:001
1489353992013,5cyclictest883-21dbus-daemon20:00:001
1489353992012,6cyclictest1522226-21ntpq20:40:181
148935399200,1cyclictest3056-21PK-Backend21:35:071
148935399200,1cyclictest3056-21PK-Backend19:22:311
148935399200,19cyclictest0-21swapper/120:24:571
1489350992018,1cyclictest0-21swapper/023:21:120
1489350992018,1cyclictest0-21swapper/022:23:070
1489350992018,1cyclictest0-21swapper/022:04:410
1489350992018,1cyclictest0-21swapper/019:49:560
1489350992018,1cyclictest0-21swapper/000:12:510
1489350992018,1cyclictest0-21swapper/000:02:030
1489350992013,6cyclictest1603498-21cron00:35:010
1489350992012,6cyclictest1534249-21cat21:15:180
1489350992012,5cyclictest436-20systemd-journal20:52:170
148935099200,19cyclictest0-21swapper/021:39:400
148935099200,19cyclictest0-21swapper/021:08:450
148935099200,19cyclictest0-21swapper/020:18:150
148935099200,19cyclictest0-21swapper/019:14:000
1489359991917,1cyclictest3056-21PK-Backend23:50:023
1489359991917,1cyclictest3056-21PK-Backend23:32:373
1489359991917,1cyclictest3056-21PK-Backend22:41:143
1489359991917,1cyclictest3056-21PK-Backend22:41:133
1489359991917,1cyclictest3056-21PK-Backend22:23:373
1489359991917,1cyclictest3056-21PK-Backend21:02:183
1489359991914,3cyclictest1576681-21strings23:15:193
1489359991914,3cyclictest1496232-21timerwakeupswit19:25:213
1489359991913,5cyclictest1518763-21ntp_states20:30:203
1489359991913,4cyclictest978-21snmpd19:51:593
1489359991913,4cyclictest978-21snmpd19:51:593
1489359991912,6cyclictest1573096-21ls23:05:163
148935999191,16cyclictest1538583-21/usr/sbin/munin21:30:013
1489359991910,7cyclictest1546625-21python321:50:213
1489359991910,7cyclictest1546625-21python321:50:213
148935999190,1cyclictest0-21swapper/319:37:353
148935999190,18cyclictest0-21swapper/322:11:343
1489358991917,1cyclictest0-21swapper/223:46:462
1489358991917,1cyclictest0-21swapper/222:50:362
1489358991917,1cyclictest0-21swapper/222:50:362
1489358991917,1cyclictest0-21swapper/219:37:242
1489358991917,1cyclictest0-21swapper/219:12:362
1489358991917,1cyclictest0-21swapper/200:26:282
148935899190,1cyclictest3056-21PK-Backend22:56:102
148935899190,18cyclictest0-21swapper/200:14:112
1489353991917,1cyclictest3056-21PK-Backend20:55:461
1489353991917,1cyclictest0-21swapper/120:29:081
1489353991917,1cyclictest0-21swapper/100:02:161
1489353991916,2cyclictest3056-21PK-Backend00:18:581
148935399190,1cyclictest3056-21PK-Backend23:16:591
148935399190,1cyclictest3056-21PK-Backend19:33:191
148935399190,18cyclictest3056-21PK-Backend23:12:341
148935399190,18cyclictest3056-21PK-Backend00:08:171
148935399190,18cyclictest0-21swapper/123:38:021
1489350991917,1cyclictest0-21swapper/023:55:400
1489350991917,1cyclictest0-21swapper/023:49:530
1489350991917,1cyclictest0-21swapper/023:12:480
1489350991917,1cyclictest0-21swapper/023:06:540
1489350991917,1cyclictest0-21swapper/022:48:080
1489350991917,1cyclictest0-21swapper/020:40:190
1489350991917,1cyclictest0-21swapper/020:31:330
1489350991917,1cyclictest0-21swapper/020:29:170
1489350991917,1cyclictest0-21swapper/020:13:420
1489350991917,1cyclictest0-21swapper/020:00:030
1489350991917,1cyclictest0-21swapper/019:38:340
1489350991912,5cyclictest1556266-21sed22:20:010
1489350991912,5cyclictest1492708-21wc19:15:210
1489350991911,7cyclictest1545494-21perf21:50:000
1489350991911,6cyclictest1559020-21sensors_temp22:25:200
1489350991911,6cyclictest1496088-21python319:25:200
148935099190,1cyclictest0-21swapper/022:52:200
148935099190,1cyclictest0-21swapper/022:52:200
148935099190,1cyclictest0-21swapper/019:57:130
148935099190,1cyclictest0-21swapper/019:57:130
148935099190,1cyclictest0-21swapper/019:41:040
148935099190,1cyclictest0-21swapper/019:33:340
148935099190,18cyclictest0-21swapper/023:54:190
148935099190,18cyclictest0-21swapper/023:38:300
148935099190,18cyclictest0-21swapper/023:31:340
148935099190,18cyclictest0-21swapper/022:30:090
148935099190,18cyclictest0-21swapper/021:29:400
148935099190,18cyclictest0-21swapper/021:04:580
148935099190,18cyclictest0-21swapper/000:35:290
15390212180,2chrt0-21swapper/021:30:120
1489359991816,1cyclictest3056-21PK-Backend22:50:273
1489359991816,1cyclictest3056-21PK-Backend22:50:273
1489359991816,1cyclictest3056-21PK-Backend20:57:183
1489359991816,1cyclictest3056-21PK-Backend20:29:333
1489359991816,1cyclictest3056-21PK-Backend00:28:413
1489359991816,1cyclictest3056-21PK-Backend00:04:213
1489359991816,1cyclictest0-21swapper/319:22:423
1489359991816,1cyclictest0-21swapper/300:07:063
1489359991815,2cyclictest3056-21PK-Backend00:31:263
148935999180,16cyclictest1579880-21hddtemp_smartct23:25:163
1489358991812,4cyclictest978-21snmpd20:04:352
1489358991810,6cyclictest1544761-21ntpq21:45:182
1489358991810,6cyclictest1492206-21cat19:15:162
148935399189,7cyclictest1564699-21users22:40:221
148935399189,7cyclictest1564699-21users22:40:221
1489353991813,3cyclictest26-21ksoftirqd/119:55:001
1489353991813,3cyclictest26-21ksoftirqd/119:55:001
1489353991813,3cyclictest15033762sleep119:47:011
1489353991811,5cyclictest1549680-21grep22:00:131
1489353991811,5cyclictest1512578-21unin-run20:15:001
1489353991810,6cyclictest1560633-21ntpq22:30:191
148935399180,5cyclictest3056-21PK-Backend21:32:091
1489350991816,1cyclictest0-21swapper/023:40:260
1489350991816,1cyclictest0-21swapper/022:44:290
1489350991816,1cyclictest0-21swapper/022:44:290
1489350991816,1cyclictest0-21swapper/019:24:350
1489350991811,6cyclictest883-21dbus-daemon20:09:590
1489350991811,5cyclictest1536096-21ntpq21:20:200
148935099181,13cyclictest1598753-21tr00:20:120
1489350991810,6cyclictest1-21systemd21:15:000
148935999179,2cyclictest42-21ksoftirqd/323:00:213
1489359991715,1cyclictest3056-21PK-Backend20:24:133
148935999170,15cyclictest1574676-21hddtemp_smartct23:10:143
1489358991715,1cyclictest0-21swapper/223:35:442
1489358991711,5cyclictest1598026-21sshd00:16:512
1489353991715,1cyclictest3056-21PK-Backend23:07:001
1489353991714,2cyclictest0-21swapper/121:05:121
1489353991712,3cyclictest1569603-21wc22:55:171
1489353991711,4cyclictest26-21ksoftirqd/120:00:131
1489353991710,5cyclictest1552153-21cat22:05:201
148935099179,6cyclictest883-21dbus-daemon23:05:010
148935099179,6cyclictest1551799-21cut22:05:160
1489350991715,1cyclictest0-21swapper/000:28:520
1489350991712,3cyclictest1580943-21sh23:30:000
1489350991711,5cyclictest1528179-21systemd-run21:00:000
148935999160,2cyclictest1538823-21apt21:30:133
1489358991611,3cyclictest1512950-21cpuspeed_turbos20:15:102
1489358991610,4cyclictest1537974-21ls21:25:222
148935099169,5cyclictest1562239-21expr22:35:120
148935099168,6cyclictest1546507-21ntpq21:50:190
148935099168,6cyclictest1546507-21ntpq21:50:190
1489350991613,2cyclictest1514553-21apt20:20:120
148935099160,1cyclictest3056-21PK-Backend21:41:230
1489359991513,1cyclictest3056-21PK-Backend23:55:153
1489359991513,1cyclictest3056-21PK-Backend23:42:533
148935399158,5cyclictest1538012-21cat21:25:211
1489350991513,1cyclictest0-21swapper/019:51:180
1489350991513,1cyclictest0-21swapper/019:51:180
1489350991510,4cyclictest1597495-21strings00:15:190
148935099148,5cyclictest1519938-21expr20:35:110
148935099141,11cyclictest1576198-21tr23:15:140
1489350991410,3cyclictest1095-21runrttasks22:10:050
148935999130,5cyclictest141rcu_preempt20:08:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional