You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-21 - 16:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot4s.osadl.org (updated Thu Aug 21, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1159558994034,4cyclictest3000063-21rs:main0
1159558994033,5cyclictest1325184-21latency_hist09:50:010
1159566993832,4cyclictest3754036-21systemd-journal12:05:202
1159566993430,2cyclictest1461874-21gpgv12:00:012
1159558993330,2cyclictest1325835-21kworker/u8:4+events_unbound10:30:020
1159568993229,2cyclictest1325228-21kworker/u8:3+flush-259:010:05:003
1159568993226,4cyclictest1288719-21latency_hist09:15:013
1159566993128,2cyclictest1258027-21kworker/u8:4+flush-259:009:35:262
1159562993024,4cyclictest3000063-21rs:main1
1159562993024,4cyclictest1221738-21latency_hist08:10:001
1159558993027,2cyclictest1423590-21kworker/u8:0+events_unbound11:34:210
1159568992924,4cyclictest1325180-21apt-get09:50:013
1159568992824,3cyclictest1357178-21/usr/sbin/munin10:20:223
1159566992824,3cyclictest1206273-21kworker/u8:2+flush-259:008:15:272
1159568992623,2cyclictest1206273-21kworker/u8:2+flush-259:008:30:003
1159566992620,4cyclictest1185247-21apt-get07:35:002
1159558992622,3cyclictest1413901-21kworker/u8:1+events_unbound11:35:250
1159568992520,3cyclictest1175220-21/usr/sbin/munin07:25:163
1159558992522,2cyclictest1185278-21kworker/u8:0+events_unbound07:49:570
1159568992420,3cyclictest1170083-21apt-get07:20:123
1159558992421,2cyclictest1367266-21kworker/u8:6+events_unbound10:53:560
1159558992420,3cyclictest1382959-21apt-get10:45:120
1159568992319,3cyclictest1424143-21base6411:25:003
1159568992317,4cyclictest3754036-21systemd-journal12:20:003
1159562992319,3cyclictest1325228-21kworker/u8:3+flush-259:010:05:561
1159562992314,8cyclictest503-21systemd-logind09:20:001
1159558992320,2cyclictest503-21systemd-logind08:05:010
1159558992319,3cyclictest1408746-21apt-get11:10:130
1159558992319,3cyclictest1258027-21kworker/u8:4+flush-259:009:20:000
1159568992217,3cyclictest3754036-21systemd-journal09:40:203
1159566992219,2cyclictest1325835-21kworker/u8:4+flush-259:011:25:262
1159566992216,4cyclictest3754036-21systemd-journal08:00:012
1159558992219,2cyclictest1325228-21kworker/u8:3+events_unbound10:59:560
1159558992219,2cyclictest1154558-21kworker/u8:2+events_unbound07:29:560
1159568992118,2cyclictest1206273-21kworker/u8:2+events_unbound08:15:193
1159568992117,3cyclictest1413901-21kworker/u8:1+flush-259:012:10:213
1159568992116,3cyclictest1299406-21cp09:25:013
1159566992118,2cyclictest3754036-21systemd-journal08:40:382
1159566992118,2cyclictest1258027-21kworker/u8:4+flush-259:009:25:002
1159562992114,5cyclictest3754036-21systemd-journal10:35:001
115956299210,1cyclictest0-21swapper/108:00:011
1159558992118,2cyclictest1456826-21kworker/u8:4+events_unbound12:35:020
1159558992118,2cyclictest1423590-21kworker/u8:0+events_unbound11:55:420
1159568992016,3cyclictest1206273-21kworker/u8:2+flush-259:008:50:003
1159568992014,5cyclictest1267284-21ntpq08:50:233
1159568992014,5cyclictest1267284-21ntpq08:50:233
1159568992014,4cyclictest1493958-21diskstats12:30:143
1159566992017,2cyclictest1367266-21kworker/u8:6+flush-259:010:35:202
1159566992014,5cyclictest1226711-21apt-get08:12:562
115956299209,9cyclictest0-21swapper/109:00:001
115956299209,1cyclictest0-21swapper/111:55:011
115956299208,1cyclictest0-21swapper/111:45:211
115956299207,2cyclictest1408477-21awk11:10:001
115956299207,2cyclictest0-21swapper/111:30:121
1159562992015,4cyclictest1154558-21kworker/u8:2+flush-259:007:30:191
1159562992015,4cyclictest1144354-21kworker/u8:3+flush-259:007:25:171
1159562992014,4cyclictest1226749-21kworker/u8:0+flush-259:008:30:261
1159562992011,8cyclictest1441000-21sed11:40:171
1159558992017,2cyclictest1423590-21kworker/u8:0+events_unbound11:20:340
1159558992017,2cyclictest1221440-21kworker/u8:1+events_unbound08:09:570
1159558992017,2cyclictest1164874-21kworker/u8:0+events_unbound07:15:130
1159558992016,3cyclictest3754036-21systemd-journal10:42:200
1159558992015,3cyclictest3754036-21systemd-journal09:10:020
1159568991916,2cyclictest1423590-21kworker/u8:0+flush-259:011:30:163
1159566991913,4cyclictest1388167-21apt-get10:50:132
1159562991914,3cyclictest1346925-21apt-get10:10:131
1159562991913,4cyclictest1299618-21cat09:25:021
1159562991911,5cyclictest1356937-21latency_hist10:20:011
1159558991916,2cyclictest1456826-21kworker/u8:4+events_unbound12:25:480
1159558991916,2cyclictest1451130-21base6411:50:020
1159558991916,2cyclictest1341522-21kworker/u8:1+events_unbound10:11:300
1159558991916,2cyclictest1325228-21kworker/u8:3+events_unbound09:53:350
1159558991916,2cyclictest1247172-21kworker/u8:1+events_unbound09:01:590
1159558991916,2cyclictest1206273-21kworker/u8:2+events_unbound08:29:140
1159558991915,3cyclictest1258027-21kworker/u8:4+events_unbound09:09:010
1159558991914,4cyclictest1440592-21apt-get11:40:110
1159566991814,3cyclictest1293795-21kworker/u8:3+flush-259:009:35:002
1159566991813,4cyclictest3754036-21systemd-journal10:25:002
1159566991812,5cyclictest1372449-21latency_hist10:35:002
115956299186,6cyclictest1472447-21awk12:10:011
1159562991814,3cyclictest1319955-21kworker/u8:0+flush-259:010:00:261
1159558991815,2cyclictest1456826-21kworker/u8:4+events_unbound12:15:100
1159558991815,2cyclictest1241994-21kworker/u8:3+events_unbound08:38:560
1159558991815,2cyclictest1221440-21kworker/u8:1+events_unbound08:13:280
1159558991815,2cyclictest1206273-21kworker/u8:2+events_unbound07:55:440
1159558991815,2cyclictest1154558-21kworker/u8:2+events_unbound07:41:330
1159558991814,3cyclictest1304894-21apt-get09:30:120
1159558991813,3cyclictest1319089-21ntp_kernel_err09:40:200
1159568991714,2cyclictest1325228-21kworker/u8:3+flush-259:011:10:153
1159568991713,3cyclictest1341522-21kworker/u8:1+flush-259:010:10:193
1159568991713,2cyclictest1288953-21/usr/sbin/munin09:15:193
1159568991712,4cyclictest3754036-21systemd-journal07:50:013
1159568991711,4cyclictest1367670-21diskstats10:30:153
1159568991711,4cyclictest1206260-21latency_hist07:55:013
1159568991711,4cyclictest1195108-21ntpq07:40:233
1159568991710,5cyclictest1461761-21latency_hist12:00:013
1159566991714,2cyclictest1319955-21kworker/u8:0+flush-259:010:15:242
1159566991713,3cyclictest0-21swapper/210:59:592
1159566991713,2cyclictest0-21swapper/208:05:012
1159562991713,3cyclictest1413901-21kworker/u8:1+events_unbound12:20:201
1159562991713,3cyclictest1325228-21kworker/u8:3+flush-259:010:40:001
1159562991713,3cyclictest1258027-21kworker/u8:4+flush-259:009:00:171
1159562991711,4cyclictest3754036-21systemd-journal09:52:201
1159562991711,4cyclictest1362340-21/usr/sbin/munin10:25:141
1159558991714,2cyclictest1413901-21kworker/u8:1+events_unbound12:02:390
1159558991714,2cyclictest1325228-21kworker/u8:3+events_unbound10:39:480
1159558991714,2cyclictest1319955-21kworker/u8:0+events_unbound09:56:190
1159558991714,2cyclictest1206273-21kworker/u8:2+events_unbound08:22:280
1159558991714,2cyclictest1185278-21kworker/u8:0+events_unbound07:38:560
1159558991714,2cyclictest1154558-21kworker/u8:2+events_unbound07:22:060
1159558991714,2cyclictest1138769-21kworker/u8:1+events_unbound07:31:400
1159568991611,4cyclictest3754036-21systemd-journal10:36:203
1159568991611,4cyclictest1413931-21apt-get11:15:143
1159568991610,4cyclictest1424515-21cat11:25:003
1159568991610,4cyclictest1216625-21latency_hist08:05:013
1159566991613,2cyclictest1456826-21kworker/u8:4+flush-259:012:20:262
1159566991613,2cyclictest1423590-21kworker/u8:0+flush-259:011:30:162
1159566991613,2cyclictest1367266-21kworker/u8:6+flush-259:011:20:252
1159566991613,2cyclictest1325228-21kworker/u8:3+flush-259:010:05:212
1159566991613,2cyclictest1226749-21kworker/u8:0+flush-259:008:30:172
1159566991612,3cyclictest1325874-21apt-get09:50:162
115956299167,1cyclictest0-21swapper/110:40:551
1159562991613,2cyclictest0-21swapper/108:20:001
1159562991612,3cyclictest1413901-21kworker/u8:1+flush-259:012:35:011
1159562991611,4cyclictest1247172-21kworker/u8:1+flush-259:008:50:171
1159562991611,4cyclictest1247172-21kworker/u8:1+flush-259:008:50:171
115955899162,4cyclictest1231259-21ntpq08:15:240
1159558991614,1cyclictest60650irq/133-enp0s29f1:rx-012:31:580
1159558991613,2cyclictest60650irq/133-enp0s29f1:rx-009:25:150
1159558991613,2cyclictest1423590-21kworker/u8:0+events_unbound11:48:100
1159558991613,2cyclictest1341522-21kworker/u8:1+events_unbound10:15:080
1159558991613,2cyclictest1341522-21kworker/u8:1+events_unbound10:08:330
1159558991613,2cyclictest1258027-21kworker/u8:4+events_unbound09:22:030
1159558991613,2cyclictest1258027-21kworker/u8:4+events_unbound08:59:360
1159558991613,2cyclictest1247172-21kworker/u8:1+events_unbound08:31:310
1159558991613,2cyclictest1144354-21kworker/u8:3+events_unbound07:52:480
1159558991613,1cyclictest60650irq/133-enp0s29f1:rx-011:25:190
1159558991612,3cyclictest1456826-21kworker/u8:4+events_unbound12:05:550
1159558991611,4cyclictest1256920-21ntpq08:40:220
1159558991611,1cyclictest191rcu_preempt10:21:100
115956899158,5cyclictest1325180-21apt-get09:50:003
1159568991513,1cyclictest0-21swapper/307:35:003
1159568991512,2cyclictest1456288-21gpgv11:55:003
1159568991512,2cyclictest1164160-21unixbench_multi07:10:263
1159568991512,2cyclictest1144354-21kworker/u8:3+flush-259:007:35:263
1159568991511,3cyclictest1206273-21kworker/u8:2+flush-259:008:10:203
1159562991512,2cyclictest1335905-21unixbench_multi09:55:251
1159562991512,2cyclictest1186359-21irqcore07:35:191
115955899152,3cyclictest1475928-21cut12:10:180
1159558991513,1cyclictest60650irq/133-enp0s29f1:rx-011:08:010
1159558991513,1cyclictest60650irq/133-enp0s29f1:rx-011:00:150
1159558991513,1cyclictest60650irq/133-enp0s29f1:rx-008:54:040
1159558991513,1cyclictest60650irq/133-enp0s29f1:rx-008:54:040
1159558991513,1cyclictest60650irq/133-enp0s29f1:rx-007:10:120
115956899148,5cyclictest1299003-21cron09:25:003
1159568991410,3cyclictest3754036-21systemd-journal12:25:213
1159568991410,3cyclictest1423590-21kworker/u8:0+flush-259:011:40:163
1159566991412,1cyclictest1498684-21cron12:35:002
1159566991412,1cyclictest1288718-21cron09:15:002
1159566991411,2cyclictest1408482-21cron11:10:002
1159566991411,2cyclictest1273315-21kworker/u8:0+flush-259:009:05:002
1159566991410,3cyclictest1185278-21kworker/u8:0+flush-259:007:45:262
1159566991410,2cyclictest0-21swapper/211:55:002
115956299149,4cyclictest3309264-21kworker/u8:4+flush-259:007:13:261
115956299148,4cyclictest1144354-21kworker/u8:3+flush-259:007:15:181
1159562991412,1cyclictest1467309-21idleruntime-cro12:05:001
1159562991411,2cyclictest0-21swapper/111:25:001
1159562991411,1cyclictest0-21swapper/109:50:001
115956299140,8cyclictest1477586-21idleruntime-cro12:15:001
1159558991413,1cyclictest60650irq/133-enp0s29f1:rx-010:29:270
1159558991412,2cyclictest211rcuc/011:15:250
1159558991412,1cyclictest0-21swapper/008:50:000
115955899140,5cyclictest16-21ksoftirqd/012:20:150
115956899136,5cyclictest3754036-21systemd-journal09:05:003
1159568991310,2cyclictest1467621-21cpuspeed_turbos12:05:103
1159568991310,1cyclictest0-21swapper/308:35:003
115956699138,4cyclictest1237669-21diskstats08:25:172
1159566991311,1cyclictest0-21swapper/210:05:002
1159566991311,1cyclictest0-21swapper/207:20:202
115956299139,3cyclictest1413901-21kworker/u8:1+flush-259:012:25:181
115956299137,4cyclictest1393465-21diskstats10:55:141
115956299137,4cyclictest1367266-21kworker/u8:6+flush-259:011:02:511
115956299136,5cyclictest3754036-21systemd-journal08:04:211
1159562991311,1cyclictest1367266-21kworker/u8:6+flush-259:011:20:001
1159562991310,2cyclictest1482315-21unixbench_singl12:15:241
1159562991310,2cyclictest1413659-21idleruntime-cro11:15:011
1159562991310,2cyclictest1284857-21kthreadcore09:10:201
115955899131,2cyclictest16-21ksoftirqd/007:10:010
1159558991311,2cyclictest61450irq/134-enp0s29f1:tx-009:35:180
115956899129,2cyclictest1383200-21fschecks_time10:45:143
115956899129,2cyclictest1319955-21kworker/u8:0+events_unbound09:55:193
115956899129,1cyclictest0-21swapper/308:55:193
115956899128,2cyclictest0-21swapper/308:10:003
115956899127,4cyclictest1206681-21diskstats07:55:143
115956899126,5cyclictest1247896-21diskstats08:35:143
115956899126,4cyclictest1325228-21kworker/u8:3+flush-259:010:05:253
1159568991211,0cyclictest0-21swapper/311:00:183
1159568991210,1cyclictest0-21swapper/307:15:263
115956699129,2cyclictest3000063-21rs:main2
115956699129,2cyclictest1336393-21systemd-run10:00:012
115956699129,2cyclictest1320454-21fschecks_count09:45:142
115956699128,3cyclictest1477835-21/usr/sbin/munin12:15:142
115956699127,4cyclictest1377815-21/usr/sbin/munin10:40:142
115956699126,4cyclictest1477596-21latency_hist12:15:012
115956299128,3cyclictest1185278-21kworker/u8:0+flush-259:007:45:191
115956299128,2cyclictest0-21swapper/111:35:151
115956299126,4cyclictest3754036-21systemd-journal10:46:211
115956899118,2cyclictest1450855-21apt-config11:50:013
115956899118,2cyclictest1388233-21cpuspeed_turbos10:50:153
115956899118,2cyclictest1232409-21cpuspeed_turbos08:20:133
115956899117,3cyclictest1483092-21cpuspeed_turbos12:20:143
115956899117,2cyclictest0-21swapper/311:05:243
115956899116,3cyclictest516-21gdbus09:10:003
115956899114,5cyclictest1462353-21cat12:00:013
115956899111,8cyclictest1367237-21latency_hist10:30:003
115956699119,1cyclictest0-21swapper/209:25:192
115956699117,3cyclictest1450094-21grep11:45:232
115956699117,2cyclictest0-21swapper/208:20:192
115956699110,9cyclictest1445486-21awk11:45:012
115956299118,2cyclictest1232409-21cpuspeed_turbos08:20:131
115956299111,9cyclictest0-21swapper/110:25:001
115956899108,1cyclictest0-21swapper/310:15:163
115956899105,3cyclictest1314568-21sshd09:35:463
115956699108,1cyclictest0-21swapper/212:25:192
115956699107,2cyclictest1247802-21cpuspeed_turbos08:35:132
115956699107,2cyclictest1247172-21kworker/u8:1+flush-259:009:40:172
115956699107,2cyclictest1206487-21kworker/u8:4+flush-259:008:05:222
115956699105,3cyclictest3000063-21rs:main2
11595689994,3cyclictest1253072-21/usr/sbin/munin08:40:153
11595669997,1cyclictest0-21swapper/209:20:002
11595669996,2cyclictest0-21swapper/207:15:002
11595669995,3cyclictest1169109-21/usr/sbin/munin07:15:242
11595629994,4cyclictest1201357-21/usr/sbin/munin07:50:161
11595669986,1cyclictest0-21swapper/207:25:202
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional