You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-20 - 05:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Wed May 20, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
179159221210,4sleep33056-21PK-Backend20:55:223
177554021100,4sleep33056-21PK-Backend20:10:193
179478321040,2sleep20-21swapper/221:05:192
139099790,61rtkit-daemon0-21swapper/019:05:180
175257226849,13sleep20-21swapper/219:07:482
175167326838,22sleep10-21swapper/119:05:171
175247626344,13sleep30-21swapper/319:06:313
18045462370,3sleep31804551-21cpuspeed_turbos21:35:123
18469662350,13sleep234-21ksoftirqd/223:35:122
139099350,3rtkit-daemon0-21swapper/120:10:581
17799332340,2sleep13056-21PK-Backend20:25:001
17653572340,4chrt3056-21PK-Backend19:40:193
1752890992811,15cyclictest1799682-21awk21:20:163
139099270,5rtkit-daemon1826164-21/usr/sbin/munin22:35:160
1752886992520,3cyclictest436-20systemd-journal21:24:562
175287899251,16cyclictest1839869-21chrt23:15:130
1752878992415,7cyclictest1787072-21cpu20:45:110
17598632230,4chrt0-21swapper/119:25:221
1752890992312,9cyclictest42-21ksoftirqd/320:25:173
175288299230,3cyclictest1846773-21/usr/sbin/munin23:35:151
175288299230,21cyclictest1788643-21eadedCompositor20:40:351
175287899232,19cyclictest1851743-21perf23:50:000
1752878992320,2cyclictest0-21swapper/021:52:540
1752878992320,2cyclictest0-21swapper/000:13:280
139099230,15rtkit-daemon3056-21PK-Backend23:46:061
139099230,14rtkit-daemon0-21swapper/120:05:371
17980542220,4chrt0-21swapper/321:15:203
175289099220,3cyclictest1858331-21sensors_temp00:05:243
1752886992220,1cyclictest0-21swapper/223:49:422
1752886992220,1cyclictest0-21swapper/223:30:162
1752886992220,1cyclictest0-21swapper/223:02:452
1752886992220,1cyclictest0-21swapper/220:15:282
1752886992218,3cyclictest0-21swapper/221:35:382
175288699220,3cyclictest978-21snmpd20:48:542
175288699220,20cyclictest0-21swapper/222:56:292
175288699220,20cyclictest0-21swapper/200:02:062
1752882992220,1cyclictest0-21swapper/122:48:381
1752882992215,6cyclictest1-21systemd23:05:001
1752882992214,6cyclictest1758508-21dump-pmu-power19:25:001
1752878992220,1cyclictest0-21swapper/023:41:510
1752878992220,1cyclictest0-21swapper/022:22:220
1752878992220,1cyclictest0-21swapper/021:49:500
1752878992218,3cyclictest0-21swapper/019:35:120
175287899221,20cyclictest13-21ksoftirqd/020:40:160
139099220,16rtkit-daemon3056-21PK-Backend21:30:401
139099220,15rtkit-daemon3056-21PK-Backend21:40:221
139099220,14rtkit-daemon0-21swapper/020:25:210
1752890992119,1cyclictest3056-21PK-Backend22:17:033
1752890992113,6cyclictest1854120-21hddtemp_smartct23:55:133
1752890992113,6cyclictest1770557-21python319:55:203
175289099210,20cyclictest3056-21PK-Backend20:34:563
175289099210,20cyclictest3056-21PK-Backend19:37:173
175289099210,20cyclictest0-21swapper/320:08:063
175289099210,19cyclictest3056-21PK-Backend23:34:023
175289099210,19cyclictest3056-21PK-Backend19:29:333
1752886992119,1cyclictest0-21swapper/223:43:542
1752886992119,1cyclictest0-21swapper/223:24:142
1752886992119,1cyclictest0-21swapper/222:42:232
1752886992119,1cyclictest0-21swapper/222:04:522
1752886992119,1cyclictest0-21swapper/221:32:342
1752886992119,1cyclictest0-21swapper/221:00:372
1752886992119,1cyclictest0-21swapper/220:26:582
1752886992119,1cyclictest0-21swapper/220:06:152
1752886992119,1cyclictest0-21swapper/220:02:102
1752886992119,1cyclictest0-21swapper/219:57:032
1752886992119,1cyclictest0-21swapper/219:36:482
1752886992119,1cyclictest0-21swapper/219:27:492
1752886992116,3cyclictest1863745-21users00:20:252
1752886992114,5cyclictest1790274-21cron20:55:002
175288699210,20cyclictest0-21swapper/222:16:142
175288699210,20cyclictest0-21swapper/219:19:112
175288699210,1cyclictest0-21swapper/219:33:452
175288299210,20cyclictest0-21swapper/120:03:581
1752878992119,1cyclictest0-21swapper/023:04:330
1752878992119,1cyclictest0-21swapper/022:57:300
1752878992119,1cyclictest0-21swapper/022:10:120
1752878992119,1cyclictest0-21swapper/021:55:590
1752878992119,1cyclictest0-21swapper/021:32:400
1752878992119,1cyclictest0-21swapper/021:11:250
1752878992119,1cyclictest0-21swapper/020:37:100
1752878992119,1cyclictest0-21swapper/020:33:050
1752878992119,1cyclictest0-21swapper/020:04:120
1752878992119,1cyclictest0-21swapper/019:52:080
1752878992119,1cyclictest0-21swapper/019:33:090
1752878992119,1cyclictest0-21swapper/019:25:250
1752878992119,1cyclictest0-21swapper/000:27:200
1752878992119,1cyclictest0-21swapper/000:06:380
1752878992119,1cyclictest0-21swapper/000:04:000
1752878992118,2cyclictest0-21swapper/021:25:440
1752878992118,2cyclictest0-21swapper/021:17:330
1752878992112,6cyclictest1095-21runrttasks22:25:200
175287899210,20cyclictest3056-21PK-Backend20:06:150
175287899210,20cyclictest0-21swapper/023:39:140
175287899210,20cyclictest0-21swapper/022:50:270
139099210,14rtkit-daemon3056-21PK-Backend20:15:471
139099210,14rtkit-daemon0-21swapper/122:58:311
139099210,14rtkit-daemon0-21swapper/022:43:530
139099210,13rtkit-daemon0-21swapper/021:35:100
1752890992018,1cyclictest3056-21PK-Backend23:06:393
1752890992018,1cyclictest3056-21PK-Backend22:51:103
1752890992018,1cyclictest3056-21PK-Backend22:04:193
1752890992018,1cyclictest0-21swapper/323:53:493
1752890992018,1cyclictest0-21swapper/319:48:053
1752890992017,2cyclictest3056-21PK-Backend19:30:343
1752890992015,3cyclictest1856942-21munin-run00:05:003
175289099201,4cyclictest1803537-21python321:30:213
175289099201,17cyclictest1862404-21munin-run00:20:003
175289099200,6cyclictest436-20systemd-journal21:10:573
175289099200,1cyclictest3056-21PK-Backend23:10:033
175289099200,19cyclictest3056-21PK-Backend22:48:333
175289099200,19cyclictest0-21swapper/321:45:193
175289099200,19cyclictest0-21swapper/321:43:173
175289099200,19cyclictest0-21swapper/321:05:383
175289099200,19cyclictest0-21swapper/320:15:173
175289099200,19cyclictest0-21swapper/300:35:133
175289099200,18cyclictest3056-21PK-Backend22:13:253
1752886992018,1cyclictest0-21swapper/222:27:362
1752886992018,1cyclictest0-21swapper/222:20:062
1752886992018,1cyclictest0-21swapper/222:05:532
1752886992018,1cyclictest0-21swapper/221:29:302
1752886992018,1cyclictest0-21swapper/221:16:112
1752886992018,1cyclictest0-21swapper/221:10:032
1752886992018,1cyclictest0-21swapper/219:52:572
1752886992018,1cyclictest0-21swapper/219:20:462
1752886992018,1cyclictest0-21swapper/200:38:092
1752886992018,1cyclictest0-21swapper/200:18:352
175288699200,1cyclictest0-21swapper/222:47:372
175288699200,19cyclictest0-21swapper/222:34:192
175288699200,19cyclictest0-21swapper/200:34:312
175288699200,19cyclictest0-21swapper/200:08:082
1752882992012,6cyclictest436-20systemd-journal23:10:031
1752882992012,6cyclictest26-21ksoftirqd/123:15:131
175288299200,19cyclictest3056-21PK-Backend00:32:211
175288299200,19cyclictest3056-21PK-Backend00:24:021
175287899202,16cyclictest2174-21JS10:27:480
1752878992018,1cyclictest0-21swapper/022:45:130
1752878992018,1cyclictest0-21swapper/021:21:380
1752878992018,1cyclictest0-21swapper/020:16:300
1752878992018,1cyclictest0-21swapper/020:11:220
1752878992018,1cyclictest0-21swapper/019:49:050
1752878992018,1cyclictest0-21swapper/019:13:420
1752878992018,1cyclictest0-21swapper/019:13:420
1752878992017,2cyclictest0-21swapper/022:00:040
175287899200,1cyclictest0-21swapper/023:10:480
175287899200,1cyclictest0-21swapper/021:43:420
175287899200,19cyclictest0-21swapper/023:55:500
175287899200,19cyclictest0-21swapper/022:30:530
175287899200,19cyclictest0-21swapper/022:16:400
175287899200,19cyclictest0-21swapper/019:16:400
175287899200,19cyclictest0-21swapper/000:37:480
175287899200,19cyclictest0-21swapper/000:22:540
139099200,14rtkit-daemon3056-21PK-Backend21:38:561
139099200,14rtkit-daemon0-21swapper/123:06:221
139099200,14rtkit-daemon0-21swapper/122:54:531
139099200,14rtkit-daemon0-21swapper/120:49:181
139099200,13rtkit-daemon0-21swapper/122:20:361
1752890991917,1cyclictest3056-21PK-Backend23:27:263
1752890991917,1cyclictest3056-21PK-Backend23:15:033
1752890991917,1cyclictest0-21swapper/321:55:343
1752890991917,1cyclictest0-21swapper/321:51:283
1752890991917,1cyclictest0-21swapper/319:52:123
1752890991917,1cyclictest0-21swapper/300:13:163
1752890991916,2cyclictest3056-21PK-Backend21:29:113
175289099190,1cyclictest3056-21PK-Backend22:06:083
175289099190,1cyclictest3056-21PK-Backend20:35:573
175289099190,1cyclictest3056-21PK-Backend20:01:103
175289099190,1cyclictest3056-21PK-Backend19:22:173
175289099190,1cyclictest0-21swapper/322:31:503
175289099190,1cyclictest0-21swapper/300:26:213
175289099190,18cyclictest3056-21PK-Backend22:23:323
175289099190,18cyclictest0-21swapper/323:24:503
175289099190,18cyclictest0-21swapper/300:24:323
175289099190,17cyclictest3056-21PK-Backend22:58:473
1752886991917,1cyclictest0-21swapper/223:19:142
1752886991917,1cyclictest0-21swapper/222:13:232
1752886991917,1cyclictest0-21swapper/220:43:472
1752886991917,1cyclictest0-21swapper/220:10:212
1752886991917,1cyclictest0-21swapper/219:47:502
1752886991912,5cyclictest1753555-21interrupts19:10:192
1752886991912,5cyclictest1753555-21interrupts19:10:192
175288699190,1cyclictest3056-21PK-Backend22:35:202
175288699190,1cyclictest0-21swapper/223:07:112
175288699190,1cyclictest0-21swapper/221:49:452
175288699190,1cyclictest0-21swapper/221:44:372
175288699190,18cyclictest0-21swapper/222:52:032
1752882991917,1cyclictest0-21swapper/119:41:071
1752882991917,1cyclictest0-21swapper/119:38:301
1752882991914,4cyclictest1791124-21missed_timers20:55:171
175288299190,1cyclictest3056-21PK-Backend23:51:241
175288299190,1cyclictest3056-21PK-Backend21:10:231
175288299190,1cyclictest0-21swapper/120:36:091
1752878991918,1cyclictest0-21swapper/021:00:560
1752878991914,3cyclictest1837874-21apt-key23:10:000
1752878991913,4cyclictest1789670-21python320:50:200
1752878991910,7cyclictest883-21dbus-daemon23:30:000
175287899190,1cyclictest0-21swapper/022:07:210
175287899190,18cyclictest0-21swapper/000:32:340
139099190,13rtkit-daemon0-21swapper/023:54:270
17792302180,3sleep20-21swapper/220:20:202
1752890991816,1cyclictest3056-21PK-Backend22:41:443
1752890991816,1cyclictest3056-21PK-Backend22:38:193
1752890991816,1cyclictest0-21swapper/319:13:243
1752890991816,1cyclictest0-21swapper/319:13:243
1752890991815,2cyclictest3056-21PK-Backend23:38:153
175289099181,4cyclictest1867259-21python300:30:233
1752890991813,3cyclictest1755786-21irqstats19:15:173
1752890991812,5cyclictest1850483-21expr23:45:133
175288699189,7cyclictest1865028-21df_inode00:25:132
175288699186,10cyclictest1855225-40ogrotate00:00:012
1752886991815,2cyclictest1095-21runrttasks20:32:172
1752886991812,5cyclictest1765563-21sed19:40:212
1752882991816,1cyclictest0-21swapper/120:53:541
1752882991813,3cyclictest1818269-21munin-run22:15:001
1752882991811,5cyclictest1766935-21awk19:45:181
1752882991810,6cyclictest1828394-21cat22:40:211
1752890991715,1cyclictest3056-21PK-Backend23:44:503
1752890991715,1cyclictest3056-21PK-Backend23:00:363
1752890991715,1cyclictest3056-21PK-Backend22:28:123
175289099171,2cyclictest1792276-21apt21:00:113
1752886991715,1cyclictest0-21swapper/223:26:512
1752886991710,6cyclictest1860059-21sensors_temp00:10:212
175288299179,6cyclictest1826111-21entropy22:35:151
1752882991715,1cyclictest0-21swapper/119:18:361
1752882991714,2cyclictest3056-21PK-Backend21:06:101
1752882991713,3cyclictest1844797-21/usr/sbin/munin23:30:001
1752882991711,5cyclictest26-21ksoftirqd/122:15:171
1752882991711,5cyclictest1849423-21sed23:40:201
1752882991710,5cyclictest1816686-21latency_hist22:10:001
175288299170,16cyclictest3056-21PK-Backend21:18:001
1752878991715,1cyclictest0-21swapper/023:24:130
1752878991715,1cyclictest0-21swapper/019:21:140
1752878991715,1cyclictest0-21swapper/000:19:290
1752878991712,3cyclictest1846525-21sh23:35:000
175287899171,11cyclictest1763964-21apt-get19:40:030
139099170,3rtkit-daemon0-21swapper/119:59:431
1752890991611,3cyclictest1778338-21/usr/sbin/munin20:20:163
1752890991610,4cyclictest42-21ksoftirqd/320:42:103
175288699169,5cyclictest1853448-21cron23:55:012
1752886991614,1cyclictest0-21swapper/221:55:052
1752886991610,4cyclictest1810485-21ls21:50:182
175288299169,5cyclictest1869579-21cron00:40:001
175288299166,9cyclictest3056-21PK-Backend00:00:011
1752882991614,1cyclictest3056-21PK-Backend23:22:521
1752882991614,1cyclictest3056-21PK-Backend21:56:551
1752882991614,1cyclictest3056-21PK-Backend19:54:321
1752882991614,1cyclictest3056-21PK-Backend00:14:221
1752882991614,1cyclictest0-21swapper/123:33:061
1752882991611,4cyclictest1783428-21sed20:35:011
1752882991610,5cyclictest1857133-21date00:05:001
1752882991610,4cyclictest1825032-21sed22:30:211
175288299160,14cyclictest3056-21PK-Backend00:17:201
175287899168,6cyclictest1791676-21sshd20:55:570
1752878991614,1cyclictest0-21swapper/021:05:160
1752878991610,4cyclictest1779946-21grep20:25:000
18092022150,4sleep13056-21PK-Backend21:45:241
1752890991511,2cyclictest42-21ksoftirqd/320:45:173
1752886991513,1cyclictest0-21swapper/223:14:142
1752886991512,2cyclictest1790677-21tr20:55:112
175288299158,6cyclictest1822716-21cat22:25:161
1752882991512,2cyclictest3056-21PK-Backend22:00:181
175288299150,14cyclictest3056-21PK-Backend00:25:381
175288299150,14cyclictest0-21swapper/119:30:051
1752890991410,2cyclictest886-21NetworkManager20:50:163
175288699149,3cyclictest1784364-21processes20:35:202
175288299146,6cyclictest1811323-21cron21:55:001
1752882991411,2cyclictest3056-21PK-Backend19:12:151
1752882991411,2cyclictest3056-21PK-Backend19:12:141
1752882991410,3cyclictest1801683-21sh21:25:181
175288299140,1cyclictest3056-21PK-Backend20:29:061
175288299140,13cyclictest3056-21PK-Backend21:21:251
175288299140,13cyclictest3056-21PK-Backend21:00:571
175288299130,6cyclictest3056-21PK-Backend00:05:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional