You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 09:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Wed May 27, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
153555924080,8sleep30-21swapper/319:05:233
153482122550,8sleep00-21swapper/019:05:120
153568922360,6sleep20-21swapper/219:05:262
165123321180,3sleep30-21swapper/300:30:213
160632921080,3sleep00-21swapper/022:25:030
160632921080,3sleep00-21swapper/022:25:030
157552421040,2sleep2331rcuc/220:55:192
153611626141,14sleep10-21swapper/119:07:521
16273972520,5sleep13056-21PK-Backend23:25:111
16365342490,3sleep11636539-21cpuspeed_turbos23:50:131
15941632480,2sleep30-21swapper/321:50:133
15520032430,3sleep30-21swapper/319:50:203
15499002420,2chrt3056-21PK-Backend19:45:121
139099380,3rtkit-daemon0-21swapper/222:57:112
139099380,3rtkit-daemon0-21swapper/221:39:212
139099380,3rtkit-daemon0-21swapper/219:14:082
139099370,3rtkit-daemon0-21swapper/200:14:272
15792652260,4sleep30-21swapper/321:05:223
15629992250,3chrt0-21swapper/120:20:251
153640699242,20cyclictest2085813-21snapd22:18:433
1536406992420,3cyclictest1635478-21smartctl23:45:203
16508822230,4sleep23056-21PK-Backend00:30:152
1536406992321,1cyclictest0-21swapper/320:40:263
153640399232,1cyclictest1633548-21strings23:40:172
1536403992319,2cyclictest3056-21PK-Backend20:50:102
1536403992314,6cyclictest1605842-21apt-get22:25:032
1536403992314,6cyclictest1605842-21apt-get22:25:032
153640399230,21cyclictest1592589-21df21:45:122
1536393992318,3cyclictest1647884-21python300:20:210
1536406992219,2cyclictest3056-21PK-Backend00:02:373
1536406992216,5cyclictest42-21ksoftirqd/321:30:183
1536403992219,2cyclictest0-21swapper/221:55:192
1536403992214,6cyclictest436-20systemd-journal20:41:282
1536397992220,1cyclictest3056-21PK-Backend19:18:061
1536397992220,1cyclictest0-21swapper/122:41:321
1536397992220,1cyclictest0-21swapper/120:51:411
153639799220,20cyclictest3056-21PK-Backend22:10:021
1536393992220,1cyclictest0-21swapper/022:55:440
153639399220,21cyclictest0-21swapper/021:55:350
139099220,14rtkit-daemon0-21swapper/319:36:083
15405422210,4sleep00-21swapper/019:20:010
1536406992119,1cyclictest3056-21PK-Backend22:45:593
1536406992119,1cyclictest3056-21PK-Backend22:14:223
1536406992119,1cyclictest3056-21PK-Backend00:39:353
1536406992119,1cyclictest0-21swapper/323:20:263
1536406992119,1cyclictest0-21swapper/323:14:043
1536406992119,1cyclictest0-21swapper/320:59:193
1536406992119,1cyclictest0-21swapper/320:13:433
1536406992119,1cyclictest0-21swapper/319:40:173
1536406992119,1cyclictest0-21swapper/300:12:443
153640699210,20cyclictest0-21swapper/322:34:163
153640699210,20cyclictest0-21swapper/320:16:403
153640699210,20cyclictest0-21swapper/319:18:473
153640699210,20cyclictest0-21swapper/300:15:283
153640699210,19cyclictest3056-21PK-Backend22:05:233
153640699210,19cyclictest3056-21PK-Backend20:25:323
153640699210,19cyclictest3056-21PK-Backend00:26:573
153640399217,5cyclictest1597848-21basename22:00:162
1536403992119,1cyclictest0-21swapper/223:59:442
1536403992119,1cyclictest0-21swapper/223:54:162
1536403992119,1cyclictest0-21swapper/223:33:062
1536403992119,1cyclictest0-21swapper/223:15:572
1536403992119,1cyclictest0-21swapper/223:11:312
1536403992119,1cyclictest0-21swapper/221:10:042
1536403992119,1cyclictest0-21swapper/221:06:252
1536403992119,1cyclictest0-21swapper/220:24:212
1536403992119,1cyclictest0-21swapper/220:05:352
1536403992119,1cyclictest0-21swapper/219:49:262
1536403992119,1cyclictest0-21swapper/200:36:002
1536403992119,1cyclictest0-21swapper/200:29:112
1536403992119,1cyclictest0-21swapper/200:19:382
1536403992114,5cyclictest1541448-21swap19:20:232
1536403992113,6cyclictest1539127-21cat19:15:122
1536403992112,7cyclictest883-21dbus-daemon22:55:002
153640399210,20cyclictest0-21swapper/222:36:282
153640399210,20cyclictest0-21swapper/220:16:032
1536397992119,1cyclictest3056-21PK-Backend20:47:151
1536397992119,1cyclictest3056-21PK-Backend20:04:361
1536397992119,1cyclictest3056-21PK-Backend19:37:261
1536397992119,1cyclictest0-21swapper/123:56:291
1536397992119,1cyclictest0-21swapper/123:47:231
1536397992119,1cyclictest0-21swapper/123:30:531
1536397992119,1cyclictest0-21swapper/120:33:231
1536397992119,1cyclictest0-21swapper/120:33:231
1536397992116,3cyclictest1644758-21sh00:15:011
1536397992114,6cyclictest1578080-21cron21:05:001
153639799210,2cyclictest1568035-21strings20:35:191
153639799210,20cyclictest0-21swapper/100:37:171
1536393992119,1cyclictest0-21swapper/023:56:350
1536393992119,1cyclictest0-21swapper/023:53:510
1536393992119,1cyclictest0-21swapper/023:46:260
1536393992119,1cyclictest0-21swapper/023:12:410
1536393992119,1cyclictest0-21swapper/023:07:410
1536393992119,1cyclictest0-21swapper/022:12:250
1536393992119,1cyclictest0-21swapper/021:41:490
1536393992119,1cyclictest0-21swapper/021:36:490
1536393992119,1cyclictest0-21swapper/021:01:000
1536393992119,1cyclictest0-21swapper/020:00:510
1536393992119,1cyclictest0-21swapper/019:39:140
1536393992119,1cyclictest0-21swapper/019:16:020
1536393992119,1cyclictest0-21swapper/000:26:220
1536393992119,1cyclictest0-21swapper/000:02:360
1536393992113,7cyclictest1574223-21users20:50:230
153639399210,20cyclictest0-21swapper/019:48:270
153639399210,20cyclictest0-21swapper/019:14:410
139099210,15rtkit-daemon3056-21PK-Backend19:31:443
139099210,14rtkit-daemon3056-21PK-Backend20:53:503
1536406992018,1cyclictest3056-21PK-Backend21:26:293
1536406992018,1cyclictest3056-21PK-Backend21:26:293
1536406992018,1cyclictest3056-21PK-Backend20:37:083
1536406992018,1cyclictest0-21swapper/322:52:073
1536406992018,1cyclictest0-21swapper/322:25:173
1536406992018,1cyclictest0-21swapper/322:25:163
1536406992018,1cyclictest0-21swapper/322:21:383
1536406992018,1cyclictest0-21swapper/321:56:243
1536406992018,1cyclictest0-21swapper/321:38:123
1536406992018,1cyclictest0-21swapper/321:11:223
1536406992018,1cyclictest0-21swapper/320:45:063
1536406992018,1cyclictest0-21swapper/320:07:003
1536406992012,6cyclictest1624259-21expr23:15:123
153640699200,1cyclictest3056-21PK-Backend00:07:173
153640699200,19cyclictest3056-21PK-Backend22:43:283
153640699200,19cyclictest0-21swapper/323:02:073
153640699200,19cyclictest0-21swapper/321:23:593
153640699200,19cyclictest0-21swapper/319:58:083
153640699200,19cyclictest0-21swapper/319:26:113
153640699200,18cyclictest1576669-21/usr/sbin/munin21:00:133
153640699200,17cyclictest1590500-21apt21:40:113
1536403992018,1cyclictest0-21swapper/222:42:032
1536403992018,1cyclictest0-21swapper/221:42:082
1536403992018,1cyclictest0-21swapper/221:32:072
1536403992018,1cyclictest0-21swapper/221:20:592
1536403992018,1cyclictest0-21swapper/220:12:182
1536403992018,1cyclictest0-21swapper/220:02:382
1536403992018,1cyclictest0-21swapper/219:50:142
1536403992018,1cyclictest0-21swapper/200:03:492
1536403992015,3cyclictest1605870-21unin-run22:25:002
1536403992013,5cyclictest1548873-21seq19:40:222
153640399200,19cyclictest0-21swapper/223:07:042
153640399200,19cyclictest0-21swapper/222:47:382
153640399200,19cyclictest0-21swapper/220:33:272
153640399200,19cyclictest0-21swapper/220:33:272
153640399200,19cyclictest0-21swapper/219:56:432
153639799202,13cyclictest1599160-21apt22:05:101
1536397992018,1cyclictest3056-21PK-Backend21:41:161
1536397992018,1cyclictest0-21swapper/119:31:251
1536397992015,3cyclictest1578495-21cpuspeed_turbos21:05:131
1536397992013,5cyclictest1583925-21df21:20:131
153639799200,19cyclictest0-21swapper/122:49:571
153639799200,19cyclictest0-21swapper/120:28:571
153639799200,19cyclictest0-21swapper/100:22:581
1536393992018,1cyclictest3056-21PK-Backend21:05:470
1536393992018,1cyclictest0-21swapper/023:17:070
1536393992018,1cyclictest0-21swapper/021:31:150
1536393992018,1cyclictest0-21swapper/021:28:450
1536393992018,1cyclictest0-21swapper/021:28:450
1536393992018,1cyclictest0-21swapper/020:25:180
1536393992018,1cyclictest0-21swapper/020:12:400
1536393992018,1cyclictest0-21swapper/019:55:440
1536393992013,6cyclictest1646494-21cron00:20:000
153639399200,1cyclictest1569426-21apt20:40:110
153639399200,1cyclictest0-21swapper/020:36:540
153639399200,1cyclictest0-21swapper/019:27:180
153639399200,19cyclictest3056-21PK-Backend22:09:210
153639399200,19cyclictest3056-21PK-Backend00:12:090
153639399200,19cyclictest0-21swapper/022:47:400
153639399200,19cyclictest0-21swapper/022:39:010
153639399200,19cyclictest0-21swapper/021:18:580
153639399200,19cyclictest0-21swapper/020:55:050
153639399200,19cyclictest0-21swapper/020:21:260
153639399200,19cyclictest0-21swapper/020:17:000
153639399200,19cyclictest0-21swapper/019:41:240
16032772190,8sleep013-21ksoftirqd/022:15:160
1536406991917,1cyclictest0-21swapper/323:35:403
1536406991917,1cyclictest0-21swapper/323:06:333
1536406991917,1cyclictest0-21swapper/322:59:373
1536406991917,1cyclictest0-21swapper/321:45:423
153640699190,1cyclictest3056-21PK-Backend20:31:543
153640699190,1cyclictest3056-21PK-Backend20:31:543
153640699190,1cyclictest3056-21PK-Backend20:21:063
153640699190,1cyclictest0-21swapper/320:03:153
153640699190,18cyclictest0-21swapper/323:43:583
153640699190,18cyclictest0-21swapper/323:34:183
1536403991917,1cyclictest0-21swapper/223:29:492
1536403991917,1cyclictest0-21swapper/221:17:542
1536403991915,3cyclictest16093832sleep222:32:512
1536403991912,6cyclictest1546535-21ntp_states19:35:182
1536403991911,6cyclictest978-21snmpd23:45:112
153640399190,1cyclictest0-21swapper/223:02:382
153640399190,1cyclictest0-21swapper/200:22:212
153640399190,18cyclictest0-21swapper/222:18:172
1536397991917,1cyclictest3056-21PK-Backend23:18:301
1536397991917,1cyclictest3056-21PK-Backend23:09:101
1536397991917,1cyclictest3056-21PK-Backend20:18:021
1536397991917,1cyclictest0-21swapper/121:31:431
1536397991916,1cyclictest3056-21PK-Backend22:20:301
1536397991915,2cyclictest3056-21PK-Backend19:55:171
1536397991913,3cyclictest1543783-21dump-pmu-power19:30:001
1536397991912,5cyclictest436-20systemd-journal22:04:551
1536397991912,5cyclictest1589693-21cat21:35:221
153639799190,1cyclictest3056-21PK-Backend23:44:171
153639799190,1cyclictest3056-21PK-Backend20:56:141
153639799190,1cyclictest0-21swapper/123:36:281
1536393991917,1cyclictest0-21swapper/023:36:470
1536393991917,1cyclictest0-21swapper/023:00:100
1536393991917,1cyclictest0-21swapper/022:53:130
1536393991917,1cyclictest0-21swapper/022:24:280
1536393991917,1cyclictest0-21swapper/021:21:280
1536393991917,1cyclictest0-21swapper/020:31:060
1536393991917,1cyclictest0-21swapper/020:31:060
1536393991917,1cyclictest0-21swapper/000:05:200
1536393991912,6cyclictest1630211-21sed23:30:210
1536393991912,5cyclictest1651240-21awk00:30:210
153639399190,1cyclictest0-21swapper/022:41:310
153639399190,1cyclictest0-21swapper/021:50:480
153639399190,1cyclictest0-21swapper/020:05:580
153639399190,1cyclictest0-21swapper/019:52:460
153639399190,1cyclictest0-21swapper/019:32:110
139099190,13rtkit-daemon0-21swapper/222:10:322
139099190,13rtkit-daemon0-21swapper/220:26:152
16225982180,3sleep10-21swapper/123:10:141
1536406991816,1cyclictest3056-21PK-Backend00:22:173
1536406991816,1cyclictest0-21swapper/322:00:013
1536406991816,1cyclictest0-21swapper/321:15:343
1536406991816,1cyclictest0-21swapper/319:45:583
1536406991816,1cyclictest0-21swapper/319:23:273
1536406991816,1cyclictest0-21swapper/319:12:253
153640699181,4cyclictest1627781-21df23:25:133
1536403991816,1cyclictest0-21swapper/223:36:252
1536403991816,1cyclictest0-21swapper/223:20:232
1536403991816,1cyclictest0-21swapper/222:07:572
1536403991816,1cyclictest0-21swapper/219:32:232
1536403991810,6cyclictest1594356-21fw_packets21:50:162
1536403991810,6cyclictest1577359-21ntp_states21:00:202
153639799189,7cyclictest436-20systemd-journal00:35:001
153639799189,7cyclictest1559900-21cron20:15:001
1536397991815,2cyclictest0-21swapper/122:17:051
1536397991811,5cyclictest1642223-21needreboot00:05:211
1536397991810,6cyclictest1585881-21wc21:25:171
1536397991810,6cyclictest1585881-21wc21:25:161
153639799180,16cyclictest3056-21PK-Backend21:55:491
1536393991816,1cyclictest0-21swapper/023:42:350
1536393991816,1cyclictest0-21swapper/023:26:330
1536393991816,1cyclictest0-21swapper/023:24:360
1536406991715,1cyclictest3056-21PK-Backend23:53:173
153640699170,15cyclictest1610156-21ls22:35:123
1536403991715,1cyclictest0-21swapper/220:48:482
1536403991715,1cyclictest0-21swapper/219:28:512
1536403991710,5cyclictest883-21dbus-daemon20:37:332
1536397991715,1cyclictest3056-21PK-Backend21:46:571
1536397991715,1cyclictest3056-21PK-Backend19:13:531
1536397991715,1cyclictest0-21swapper/100:03:381
1536397991714,2cyclictest3056-21PK-Backend19:53:551
1536397991712,3cyclictest1607837-21idleruntime-cro22:30:001
1536397991712,3cyclictest1607837-21idleruntime-cro22:30:001
1536397991711,4cyclictest1557153-21cat20:05:161
153639799170,1cyclictest3056-21PK-Backend19:21:101
1536393991715,1cyclictest0-21swapper/022:31:110
1536393991715,1cyclictest0-21swapper/021:14:460
1536393991715,1cyclictest0-21swapper/020:46:130
1536393991715,1cyclictest0-21swapper/000:38:260
1536403991612,3cyclictest1641539-21/usr/sbin/munin00:05:242
153639799169,5cyclictest1547749-21seq19:40:021
153639799163,9cyclictest1648754-21apt00:25:091
1536397991614,1cyclictest3056-21PK-Backend22:59:511
1536397991614,1cyclictest3056-21PK-Backend21:17:031
1536397991614,1cyclictest3056-21PK-Backend21:14:121
153639799161,11cyclictest1568890-21apt-get20:40:031
153639399168,7cyclictest1598758-21sshd22:03:290
1536406991513,1cyclictest3056-21PK-Backend23:59:193
1536403991513,1cyclictest0-21swapper/221:26:332
1536403991513,1cyclictest0-21swapper/221:26:332
153639799159,4cyclictest1615620-21grep22:50:181
1536397991513,1cyclictest3056-21PK-Backend00:18:321
153639799150,6cyclictest3056-21PK-Backend22:36:041
1536393991510,3cyclictest1592872-21needreboot21:45:170
153639799146,6cyclictest1620218-21cron23:05:001
153639799140,1cyclictest3056-21PK-Backend21:54:071
153639799139,3cyclictest3056-21PK-Backend23:24:101
153639799139,3cyclictest3056-21PK-Backend22:32:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional