You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 08:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Tue May 05, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
164067927639,31sleep10-21swapper/119:08:421
163969726743,15sleep20-21swapper/219:05:162
164076826545,13sleep30-21swapper/319:09:513
164011826542,14sleep00-21swapper/019:05:260
16642952620,4sleep13056-21PK-Backend20:15:011
16769852610,15sleep013-21ksoftirqd/020:50:000
16761882500,4sleep13056-21PK-Backend20:45:201
16583602320,2sleep10-21swapper/119:55:211
139099320,4rtkit-daemon1389-21rtkit-daemon23:22:321
17168192300,3sleep00-21swapper/022:40:200
17168192300,3sleep00-21swapper/022:40:200
139099280,4rtkit-daemon1389-21rtkit-daemon22:39:051
164100099240,22cyclictest1664979-21cat20:15:173
164099599241,21cyclictest1753924-21apt00:30:012
17313782230,13sleep2341ksoftirqd/223:25:012
1640995992315,6cyclictest1704813-21cron22:10:002
1640987992318,3cyclictest1662445-21dump-pmu-power20:10:010
17522102220,4chrt0-21swapper/300:25:003
17179982220,4sleep33056-21PK-Backend22:45:163
164100099227,13cyclictest1692792-21cpuspeed21:35:123
1641000992220,1cyclictest3056-21PK-Backend23:26:543
1641000992220,1cyclictest0-21swapper/320:54:173
164100099220,20cyclictest3056-21PK-Backend23:49:313
164100099220,20cyclictest3056-21PK-Backend22:25:093
1640995992220,1cyclictest0-21swapper/222:01:502
1640995992220,1cyclictest0-21swapper/219:30:152
1640995992213,7cyclictest1710916-21df22:25:142
1640992992217,3cyclictest1685760-21find21:15:111
1640987992220,1cyclictest0-21swapper/023:58:180
1640987992220,1cyclictest0-21swapper/022:20:300
1640987992220,1cyclictest0-21swapper/019:34:010
1640987992220,1cyclictest0-21swapper/000:08:250
164098799220,21cyclictest0-21swapper/000:26:490
139099220,14rtkit-daemon0-21swapper/019:37:430
164100099217,12cyclictest42-21ksoftirqd/320:35:163
1641000992119,1cyclictest3056-21PK-Backend23:42:153
1641000992119,1cyclictest3056-21PK-Backend23:42:153
1641000992119,1cyclictest3056-21PK-Backend22:05:283
1641000992119,1cyclictest0-21swapper/323:08:083
1641000992119,1cyclictest0-21swapper/321:57:513
1641000992119,1cyclictest0-21swapper/321:54:263
1641000992119,1cyclictest0-21swapper/321:34:123
1641000992119,1cyclictest0-21swapper/320:59:173
1641000992119,1cyclictest0-21swapper/300:17:023
1641000992118,2cyclictest3056-21PK-Backend23:56:003
1641000992118,2cyclictest3056-21PK-Backend19:57:053
164100099210,20cyclictest3056-21PK-Backend22:36:383
164100099210,20cyclictest0-21swapper/323:11:463
164100099210,20cyclictest0-21swapper/321:29:123
164100099210,20cyclictest0-21swapper/320:24:293
164100099210,19cyclictest3056-21PK-Backend23:31:203
164100099210,19cyclictest3056-21PK-Backend21:22:503
1640995992119,1cyclictest0-21swapper/220:58:032
1640995992119,1cyclictest0-21swapper/220:54:172
1640995992119,1cyclictest0-21swapper/220:49:442
1640995992119,1cyclictest0-21swapper/220:20:172
1640995992119,1cyclictest0-21swapper/200:22:372
1640995992113,6cyclictest1746962-21cron00:10:002
1640992992119,1cyclictest0-21swapper/123:33:201
1640992992117,2cyclictest1642839-21sh19:15:001
1640992992113,5cyclictest1710289-21idleruntime-cro22:25:001
1640992992111,7cyclictest436-20systemd-journal23:20:001
1640992992111,7cyclictest1708557-21ls22:20:001
164099299210,1cyclictest3056-21PK-Backend00:01:051
164099299210,1cyclictest0-21swapper/122:10:131
164099299210,19cyclictest3056-21PK-Backend20:18:051
1640987992119,1cyclictest0-21swapper/021:59:480
1640987992119,1cyclictest0-21swapper/021:11:010
1640987992119,1cyclictest0-21swapper/020:32:010
1640987992119,1cyclictest0-21swapper/019:54:090
1640987992119,1cyclictest0-21swapper/019:54:090
1640987992119,1cyclictest0-21swapper/019:42:000
1640987992119,1cyclictest0-21swapper/019:11:440
164098799210,20cyclictest0-21swapper/023:37:500
164098799210,20cyclictest0-21swapper/023:21:410
164098799210,20cyclictest0-21swapper/022:06:380
164098799210,20cyclictest0-21swapper/021:33:330
164098799210,20cyclictest0-21swapper/021:01:010
164100099204,14cyclictest1697466-21timerwakeupswit21:45:213
1641000992018,1cyclictest3056-21PK-Backend22:41:043
1641000992018,1cyclictest3056-21PK-Backend22:41:043
1641000992018,1cyclictest3056-21PK-Backend22:19:073
1641000992018,1cyclictest3056-21PK-Backend20:27:063
1641000992018,1cyclictest3056-21PK-Backend19:25:563
1641000992018,1cyclictest3056-21PK-Backend19:16:443
1641000992018,1cyclictest0-21swapper/322:00:283
1641000992013,5cyclictest1744519-21ntp_states00:00:203
1641000992012,6cyclictest42-21ksoftirqd/322:55:383
164100099200,1cyclictest3056-21PK-Backend23:00:513
164100099200,1cyclictest3056-21PK-Backend21:19:393
164100099200,19cyclictest3056-21PK-Backend22:34:493
164100099200,19cyclictest3056-21PK-Backend19:44:073
164100099200,19cyclictest3056-21PK-Backend19:11:223
164100099200,19cyclictest0-21swapper/320:43:433
164100099200,19cyclictest0-21swapper/320:05:163
164100099200,19cyclictest0-21swapper/300:05:063
1640995992018,1cyclictest0-21swapper/220:32:342
1640995992018,1cyclictest0-21swapper/220:17:552
1640995992018,1cyclictest0-21swapper/219:11:362
164099599200,8cyclictest16751422cat20:45:002
164099599200,19cyclictest0-21swapper/223:04:302
1640992992018,1cyclictest3056-21PK-Backend21:59:591
1640992992018,1cyclictest0-21swapper/119:41:081
164099299200,1cyclictest0-21swapper/123:03:401
164099299200,1cyclictest0-21swapper/122:08:241
164099299200,19cyclictest3056-21PK-Backend00:35:261
164099299200,19cyclictest0-21swapper/120:36:441
164099299200,19cyclictest0-21swapper/120:34:071
164099299200,19cyclictest0-21swapper/119:37:101
164099299200,19cyclictest0-21swapper/119:33:581
164099299200,18cyclictest3056-21PK-Backend00:29:581
164099299200,18cyclictest3056-21PK-Backend00:10:111
1640987992018,1cyclictest0-21swapper/023:13:360
1640987992018,1cyclictest0-21swapper/022:25:440
1640987992018,1cyclictest0-21swapper/022:17:530
1640987992018,1cyclictest0-21swapper/021:51:230
1640987992018,1cyclictest0-21swapper/021:36:090
1640987992018,1cyclictest0-21swapper/021:06:010
1640987992018,1cyclictest0-21swapper/020:29:240
1640987992018,1cyclictest0-21swapper/020:21:470
1640987992018,1cyclictest0-21swapper/020:18:220
1640987992018,1cyclictest0-21swapper/019:45:580
1640987992015,3cyclictest1697897-21sh21:50:000
1640987992013,5cyclictest1733539-21expr23:30:130
1640987992013,5cyclictest1714290-21ls22:35:110
1640987992012,6cyclictest1643094-21cat19:15:010
164098799200,19cyclictest0-21swapper/023:42:300
164098799200,19cyclictest0-21swapper/023:42:300
164098799200,19cyclictest0-21swapper/022:30:100
164098799200,19cyclictest0-21swapper/022:12:390
164098799200,19cyclictest0-21swapper/021:19:470
164098799200,19cyclictest0-21swapper/000:16:430
17307482190,4chrt3056-21PK-Backend23:20:203
164100099197,7cyclictest42-21ksoftirqd/323:15:213
1641000991917,1cyclictest0-21swapper/319:20:213
164100099191,4cyclictest1663582-21python320:10:203
1641000991914,3cyclictest1695073-21df_inode21:40:143
1641000991912,6cyclictest1681110-21sed21:00:173
1641000991912,5cyclictest436-20systemd-journal21:10:213
1641000991912,5cyclictest1752435-21/usr/sbin/munin00:25:173
1641000991910,7cyclictest42-21ksoftirqd/323:40:003
164100099190,1cyclictest0-21swapper/300:33:253
164100099190,1cyclictest0-21swapper/300:14:123
1640995991917,1cyclictest0-21swapper/221:15:132
1640995991914,4cyclictest34-21ksoftirqd/219:48:212
1640995991914,3cyclictest978-21snmpd00:13:192
1640995991914,3cyclictest34-21ksoftirqd/222:15:012
1640995991912,5cyclictest34-21ksoftirqd/223:05:162
1640995991911,6cyclictest436-20systemd-journal20:25:042
164099599190,1cyclictest0-21swapper/223:15:182
164099599190,1cyclictest0-21swapper/219:50:162
164099599190,1cyclictest0-21swapper/219:50:162
1640992991917,1cyclictest3056-21PK-Backend21:29:101
1640992991917,1cyclictest3056-21PK-Backend21:01:521
1640992991917,1cyclictest3056-21PK-Backend20:22:181
1640992991917,1cyclictest3056-21PK-Backend00:23:431
1640992991917,1cyclictest0-21swapper/123:52:471
1640992991917,1cyclictest0-21swapper/123:44:291
1640992991917,1cyclictest0-21swapper/123:44:291
1640992991917,1cyclictest0-21swapper/123:29:421
1640992991917,1cyclictest0-21swapper/120:40:091
1640992991917,1cyclictest0-21swapper/119:54:531
1640992991917,1cyclictest0-21swapper/119:54:531
1640992991914,3cyclictest1895067-21snapd23:46:271
1640992991912,5cyclictest1-21systemd20:05:001
164099299190,1cyclictest3056-21PK-Backend23:10:561
164099299190,1cyclictest3056-21PK-Backend20:05:421
164099299190,1cyclictest3056-21PK-Backend00:31:471
164099299190,1cyclictest0-21swapper/122:58:121
164099299190,1cyclictest0-21swapper/122:50:081
164099299190,1cyclictest0-21swapper/122:28:311
164099299190,1cyclictest0-21swapper/121:11:391
164099299190,1cyclictest0-21swapper/120:27:191
1640987991917,1cyclictest0-21swapper/023:27:560
1640987991917,1cyclictest0-21swapper/023:19:520
1640987991917,1cyclictest0-21swapper/023:06:200
1640987991917,1cyclictest0-21swapper/023:00:520
1640987991917,1cyclictest0-21swapper/022:51:460
1640987991917,1cyclictest0-21swapper/022:48:080
1640987991917,1cyclictest0-21swapper/000:21:230
139099190,13rtkit-daemon0-21swapper/019:21:110
164100099185,11cyclictest1660748-21unin-run20:05:003
1641000991816,1cyclictest3056-21PK-Backend22:12:183
1641000991816,1cyclictest0-21swapper/300:36:163
1641000991813,3cyclictest1676874-21munin-run20:50:003
164099599189,7cyclictest436-20systemd-journal19:45:002
1640995991816,1cyclictest0-21swapper/223:59:252
1640995991816,1cyclictest0-21swapper/223:46:142
1640995991816,1cyclictest0-21swapper/223:40:262
1640995991816,1cyclictest0-21swapper/223:40:262
1640995991816,1cyclictest0-21swapper/222:53:552
1640995991816,1cyclictest0-21swapper/222:20:292
1640995991816,1cyclictest0-21swapper/221:34:532
1640995991816,1cyclictest0-21swapper/221:14:262
1640995991816,1cyclictest0-21swapper/220:12:332
1640995991816,1cyclictest0-21swapper/219:56:112
1640995991814,3cyclictest978-21snmpd21:53:482
1640995991811,6cyclictest1756951-21sed00:35:202
1640995991811,5cyclictest1698123-21sed21:50:012
1640995991811,5cyclictest1662453-21/usr/sbin/munin20:10:002
1640995991811,5cyclictest1660911-21wc20:05:012
164099599180,1cyclictest0-21swapper/222:17:182
1640992991816,1cyclictest0-21swapper/123:36:111
1640992991816,1cyclictest0-21swapper/119:21:281
1640992991810,6cyclictest1688222-21ntpq21:20:191
1640992991810,6cyclictest1644345-21ntpq19:15:191
164099299180,1cyclictest0-21swapper/122:46:291
1640987991816,1cyclictest0-21swapper/022:57:140
1640987991816,1cyclictest0-21swapper/022:02:250
1640987991816,1cyclictest0-21swapper/020:38:030
1640987991816,1cyclictest0-21swapper/020:14:100
1640987991812,5cyclictest1739120-21sed23:45:160
1640987991812,5cyclictest1694762-21cat21:40:120
1640987991811,5cyclictest1687764-21expr21:20:120
1640987991810,6cyclictest1647611-21sed19:25:160
164098799180,6cyclictest1678579-21perf20:55:000
16483682171,5sleep234-21ksoftirqd/219:28:042
164100099179,7cyclictest3056-21PK-Backend19:40:003
164100099172,9cyclictest16822871/usr/sbin/munin21:05:223
1641000991715,1cyclictest3056-21PK-Backend23:50:333
1641000991715,1cyclictest3056-21PK-Backend22:21:443
1641000991715,1cyclictest3056-21PK-Backend19:33:123
164099599179,6cyclictest1689540-21expr21:25:132
164099599172,13cyclictest2175-21JS10:27:482
164099599172,11cyclictest1680578-21apt21:00:122
1640995991715,1cyclictest0-21swapper/221:36:292
1640995991715,1cyclictest0-21swapper/221:05:332
1640995991715,1cyclictest0-21swapper/219:35:092
1640995991714,2cyclictest1095-21runrttasks21:55:412
1640995991712,3cyclictest1713228-21xargs22:30:202
1640995991711,4cyclictest1645749-21basename19:20:162
1640995991710,6cyclictest1696376-21sed21:45:002
1640995991710,5cyclictest1744488-21ntpq00:00:202
164099599170,7cyclictest1741754-21dump-pmu-power23:55:002
164099599170,16cyclictest0-21swapper/223:36:132
164099299179,6cyclictest436-20systemd-journal19:30:001
164099299177,8cyclictest1703085-21cron22:05:001
1640992991715,1cyclictest3056-21PK-Backend23:06:301
1640992991715,1cyclictest3056-21PK-Backend21:49:111
1640992991715,1cyclictest3056-21PK-Backend00:17:271
1640992991715,1cyclictest0-21swapper/123:55:381
1640992991715,1cyclictest0-21swapper/122:42:511
1640992991715,1cyclictest0-21swapper/122:42:511
1640992991715,1cyclictest0-21swapper/122:32:101
1640992991710,6cyclictest1683855-21ls21:10:011
1640987991715,1cyclictest0-21swapper/000:11:160
1640987991714,2cyclictest3056-21PK-Backend00:35:090
1640987991712,3cyclictest1745245-21turbostat00:05:000
1640987991711,5cyclictest1674166-21sed20:40:160
1641000991614,1cyclictest3056-21PK-Backend22:52:473
1641000991614,1cyclictest3056-21PK-Backend20:30:313
164099599168,6cyclictest1754904-21ntpq00:30:182
1640995991614,1cyclictest0-21swapper/222:48:212
164099599161,12cyclictest1727280-21python323:10:202
164099599160,15cyclictest0-21swapper/222:39:222
164099599160,14cyclictest883-21dbus-daemon23:00:002
1640992991614,1cyclictest3056-21PK-Backend21:44:101
1640992991614,1cyclictest3056-21PK-Backend00:08:221
164098799169,5cyclictest1657910-21fw_packets19:55:150
1640987991611,3cyclictest1095-21runrttasks00:31:210
1640987991610,5cyclictest1679427-21wc20:55:170
139099160,13rtkit-daemon0-21swapper/121:30:481
164099599159,5cyclictest1733976-21wc23:30:182
164099599159,4cyclictest1644219-21grep19:15:172
164099599151,11cyclictest1749732-21ntp_states00:15:202
1640992991513,1cyclictest3056-21PK-Backend21:54:121
1640992991513,1cyclictest3056-21PK-Backend21:39:101
1640987991512,2cyclictest1690706-21sh21:30:000
1640987991511,3cyclictest1740271-21acpi23:50:100
1640987991510,4cyclictest1659983-21ntpq20:00:190
164099599149,4cyclictest1672767-21python320:35:192
164099599149,3cyclictest1716156-21idleruntime22:40:132
164099599149,3cyclictest1716156-21idleruntime22:40:132
164099299141,2cyclictest141rcu_preempt20:55:591
1640992991411,2cyclictest1654740-21ntp_states19:45:181
1641000991310,2cyclictest3056-21PK-Backend19:47:053
164099599137,4cyclictest34-21ksoftirqd/223:25:392
164099299131,2cyclictest141rcu_preempt20:50:141
164100099120,1cyclictest3056-21PK-Backend19:51:133
164100099120,1cyclictest3056-21PK-Backend19:51:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional