You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 14:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Dec 28, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1297856994240,1cyclictest132-1kworker/2:1H+kblockd23:47:112
1297856993529,4cyclictest1631327-21latency_hist00:30:012
1297860993424,6cyclictest1314749-21/usr/sbin/munin19:25:193
1297852993428,4cyclictest1438122-21ntpq21:20:241
1297860993330,2cyclictest3425181-21kworker/3:0+pm23:47:123
1297850993227,3cyclictest1423768-21apt-get21:10:110
1297856993127,3cyclictest1407860-21kworker/u8:4+flush-259:022:20:252
1297850993023,5cyclictest1443271-21ntpq21:25:220
1297860992921,5cyclictest1640713-21ntpq00:35:223
1297856992925,2cyclictest1345413-21kworker/u8:2+flush-259:020:00:222
1297852992926,2cyclictest1511692-21kworker/u8:2+flush-259:022:40:261
1297852992926,2cyclictest1443468-21kworker/u8:2+events_unbound22:15:001
1297852992925,3cyclictest1537409-21kworker/u8:1+events_unbound23:47:101
1297856992824,3cyclictest1423027-21kworker/u8:1+events_unbound21:20:242
1297852992825,2cyclictest1381472-21kworker/u8:0+flush-259:022:10:001
1297850992822,4cyclictest252-21systemd-journal21:20:000
1297856992724,2cyclictest1345413-21kworker/u8:2+flush-259:020:35:162
1297860992618,5cyclictest1485805-21latency_hist22:10:013
1297856992623,2cyclictest1297826-21kworker/u8:2+flush-259:019:35:222
1297852992623,2cyclictest1260544-21kworker/u8:1+flush-259:019:20:001
1297850992623,2cyclictest1443468-21kworker/u8:2+events_unbound21:50:190
1297850992623,2cyclictest1297826-21kworker/u8:2+events_unbound19:35:220
1297850992622,3cyclictest1511692-21kworker/u8:2+flush-259:023:10:160
1297860992519,4cyclictest252-21systemd-journal22:21:303
1297852992521,3cyclictest1277441-21kworker/u8:3+events_unbound19:40:151
1297856992418,4cyclictest1307604-21ntpq19:15:232
1297852992421,2cyclictest1443468-21kworker/u8:2+flush-259:022:00:251
1297852992421,2cyclictest1345413-21kworker/u8:2+events_unbound20:35:151
1297860992318,4cyclictest252-21systemd-journal20:55:003
1297852992320,2cyclictest1594546-21kworker/u8:0+events_unbound00:20:151
1297852992320,2cyclictest1381472-21kworker/u8:0+events_unbound21:15:161
1297852992319,3cyclictest1527424-21kworker/u8:3+flush-259:022:55:121
1297852992318,4cyclictest1532567-21latency_hist22:55:011
1297852992317,4cyclictest1541914-21ntpq23:00:211
1297850992320,2cyclictest1443468-21kworker/u8:2+flush-259:022:05:250
1297850992319,3cyclictest1543045-21kworker/u8:3+events_unbound23:15:230
129786099220,10cyclictest1434189-21/usr/sbin/munin21:20:163
1297856992219,2cyclictest1594546-21kworker/u8:0+events_unbound00:15:112
1297856992218,3cyclictest1512228-21apt-get22:35:122
1297856992218,3cyclictest1361137-21apt-get20:10:132
1297856992218,3cyclictest1345413-21kworker/u8:2+events_unbound20:15:222
1297856992218,3cyclictest1277441-21kworker/u8:3+flush-259:019:55:212
1297856992217,4cyclictest1599469-21latency_hist00:00:012
1297852992219,2cyclictest1511692-21kworker/u8:2+flush-259:000:35:001
1297852992219,2cyclictest1308518-21kworker/u8:0+events_unbound19:25:141
1297852992218,3cyclictest1391448-21kworker/u8:3+flush-259:020:45:231
1297852992217,3cyclictest252-21systemd-journal20:07:291
1297850992219,2cyclictest1345413-21kworker/u8:2+flush-259:021:05:180
1297850992218,3cyclictest1537409-21kworker/u8:1+events_unbound23:25:250
1297850992218,3cyclictest1423027-21kworker/u8:1+flush-259:021:20:190
1297850992218,2cyclictest1308518-21kworker/u8:0+flush-259:019:25:170
1297860992114,5cyclictest1573741-21latency_hist23:35:013
1297860992110,9cyclictest252-21systemd-journal23:39:593
129786099210,10cyclictest1626425-21/usr/sbin/munin00:25:263
1297856992117,3cyclictest252-21systemd-journal22:35:002
1297856992117,3cyclictest1277441-21kworker/u8:3+events_unbound19:40:192
1297852992118,2cyclictest1511692-21kworker/u8:2+events_unbound23:50:111
1297852992118,2cyclictest1511692-21kworker/u8:2+events_unbound23:15:251
1297852992116,4cyclictest252-21systemd-journal21:50:281
1297850992118,3cyclictest1511692-21kworker/u8:2+events_unbound23:50:220
1297850992118,2cyclictest1345413-21kworker/u8:2+flush-259:020:35:170
1297850992118,1cyclictest0-21swapper/019:20:000
1297850992117,3cyclictest1594546-21kworker/u8:0+events_unbound00:20:130
1297860992013,5cyclictest1319891-21/usr/sbin/munin19:30:153
1297860992012,7cyclictest0-21swapper/320:45:003
129786099200,10cyclictest1453659-21modprobe21:35:243
129786099200,10cyclictest1453659-21modprobe21:35:243
1297856992017,3cyclictest1308518-21kworker/u8:0+events_unbound19:20:252
1297856992017,2cyclictest1443468-21kworker/u8:2+flush-259:021:35:262
1297856992017,2cyclictest1443468-21kworker/u8:2+flush-259:021:35:252
1297856992017,2cyclictest1345413-21kworker/u8:2+flush-259:020:05:192
1297856992016,3cyclictest1345413-21kworker/u8:2+flush-259:021:05:132
1297852992017,2cyclictest1407860-21kworker/u8:4+flush-259:022:20:271
1297852992014,4cyclictest1567758-21ntpq23:25:271
1297850992017,2cyclictest1537409-21kworker/u8:1+flush-259:023:45:300
1297850992017,2cyclictest1277441-21kworker/u8:3+flush-259:019:55:250
129786099198,9cyclictest1314415-21cat19:25:013
129786099191,15cyclictest1-21systemd22:55:013
129786099190,15cyclictest1464477-21chrt21:46:503
1297856991916,2cyclictest1537409-21kworker/u8:1+events_unbound23:43:032
1297856991916,2cyclictest1345413-21kworker/u8:2+flush-259:020:55:242
1297856991915,3cyclictest1594546-21kworker/u8:0+events_unbound00:35:142
1297856991915,3cyclictest1527424-21kworker/u8:3+events_unbound22:55:172
1297856991915,3cyclictest1511692-21kworker/u8:2+flush-259:000:20:222
1297856991915,3cyclictest1449117-21journal-offline21:32:302
1297856991915,3cyclictest1371434-21apt-get20:20:122
1297856991913,4cyclictest1527405-21latency_hist22:50:022
1297852991917,1cyclictest1496546-21kworker/u8:0+flush-259:022:35:241
1297852991916,2cyclictest1543045-21kworker/u8:3+events_unbound23:10:211
1297852991916,2cyclictest1443468-21kworker/u8:2+flush-259:022:15:221
1297852991916,2cyclictest1329900-21kworker/u8:0+flush-259:019:51:291
1297852991916,2cyclictest1308518-21kworker/u8:0+flush-259:019:20:211
1297850991917,2cyclictest1335133-21kworker/u8:1+events_unbound20:25:180
1297850991916,2cyclictest509-21rs:main0
1297850991916,2cyclictest1594546-21kworker/u8:0+flush-259:000:15:150
1297850991916,2cyclictest1594546-21kworker/u8:0+events_unbound00:30:250
1297850991916,2cyclictest1496546-21kworker/u8:0+flush-259:022:40:220
1297850991916,2cyclictest1345413-21kworker/u8:2+events_unbound20:55:170
1297850991916,2cyclictest1329900-21kworker/u8:0+events_unbound19:51:320
1297850991915,3cyclictest1443468-21kworker/u8:2+flush-259:021:45:270
1297850991915,3cyclictest1335133-21kworker/u8:1+events_unbound20:10:170
1297850991915,3cyclictest1298044-21apt-get19:10:140
129786099188,8cyclictest0-21swapper/300:00:273
129786099185,10cyclictest1615798-21sed00:15:153
1297860991815,2cyclictest1345413-21kworker/u8:2+flush-259:020:35:193
1297860991815,1cyclictest0-21swapper/321:10:003
129786099181,15cyclictest1340397-21date19:50:013
1297860991811,4cyclictest1459621-21latency_hist21:45:003
129786099180,9cyclictest834-21snmpd23:20:263
129786099180,9cyclictest1381279-21taskset20:28:083
129786099180,15cyclictest1434024-21cat21:20:013
129786099180,14cyclictest1511771-21chrt22:32:503
129786099180,10cyclictest1552949-21seq23:13:103
129786099180,10cyclictest1412779-21chrt20:59:463
129785699189,5cyclictest1443468-21kworker/u8:2+events_unbound22:05:252
1297856991815,2cyclictest1537409-21kworker/u8:1+flush-259:023:35:302
1297856991815,2cyclictest1277441-21kworker/u8:3+events_unbound19:53:292
1297856991814,3cyclictest1511692-21kworker/u8:2+flush-259:000:00:242
1297852991815,2cyclictest2971-21cron00:30:001
1297852991815,2cyclictest1407860-21kworker/u8:4+events_unbound22:25:251
1297852991815,2cyclictest1346061-21irqcore19:55:171
1297852991814,3cyclictest1407860-21kworker/u8:4+flush-259:022:30:211
1297852991813,4cyclictest252-21systemd-journal00:10:011
129785099189,2cyclictest1297826-21kworker/u8:2+events_unbound19:30:180
1297850991815,2cyclictest1537409-21kworker/u8:1+flush-259:023:20:180
1297850991815,2cyclictest1381472-21kworker/u8:0+events_unbound21:55:230
1297850991814,3cyclictest1381472-21kworker/u8:0+events_unbound21:35:010
1297850991814,3cyclictest1381472-21kworker/u8:0+events_unbound21:35:010
129786099176,9cyclictest490-21dbus-daemon23:00:013
129786099171,8cyclictest1625488-21runrttasks00:21:523
129786099171,7cyclictest1361112-21/usr/sbin/munin20:10:163
1297860991713,3cyclictest1443468-21kworker/u8:2+flush-259:021:50:193
129786099171,14cyclictest1489476-21kthreadcore22:10:193
129786099170,9cyclictest1402092-21seq20:45:533
129786099170,9cyclictest1329617-21chrt19:36:083
129786099170,9cyclictest1302676-21taskset19:11:443
129786099170,8cyclictest1417554-21taskset21:00:293
129786099170,7cyclictest1543053-21/usr/sbin/munin23:05:243
129786099170,14cyclictest1335306-21latency_hist19:45:013
1297856991715,2cyclictest1594546-21kworker/u8:0+flush-259:000:30:222
1297856991714,2cyclictest1557807-21unixbench_multi23:15:272
1297856991714,2cyclictest1537409-21kworker/u8:1+flush-259:023:30:212
1297856991714,2cyclictest1537409-21kworker/u8:1+flush-259:023:20:152
1297856991714,2cyclictest1381472-21kworker/u8:0+events_unbound21:56:292
1297856991714,2cyclictest1314841-21cpuspeed_turbos19:25:112
1297852991714,2cyclictest1594546-21kworker/u8:0+events_unbound00:15:231
1297852991714,2cyclictest1543045-21kworker/u8:3+flush-259:023:05:221
1297852991714,2cyclictest1537409-21kworker/u8:1+flush-259:023:40:311
1297852991714,2cyclictest1537409-21kworker/u8:1+events_unbound23:20:151
1297852991714,2cyclictest1511692-21kworker/u8:2+flush-259:022:45:141
1297852991714,2cyclictest1464756-21sh21:50:001
1297852991714,2cyclictest1381472-21kworker/u8:0+flush-259:020:40:211
1297852991714,2cyclictest1345413-21kworker/u8:2+flush-259:020:15:201
1297852991714,2cyclictest1345413-21kworker/u8:2+flush-259:020:10:211
1297852991714,2cyclictest1345413-21kworker/u8:2+events_unbound20:20:241
1297852991714,2cyclictest1335133-21kworker/u8:1+flush-259:020:25:271
1297852991714,2cyclictest1297826-21kworker/u8:2+flush-259:019:35:181
1297852991713,3cyclictest1381472-21kworker/u8:0+flush-259:021:00:001
1297852991713,3cyclictest1381472-21kworker/u8:0+events_unbound21:10:201
1297852991713,3cyclictest1345413-21kworker/u8:2+flush-259:020:55:261
1297850991714,2cyclictest1537409-21kworker/u8:1+events_unbound23:00:220
1297850991714,2cyclictest1527424-21kworker/u8:3+flush-259:022:53:300
1297850991714,2cyclictest1501940-21kworker/u8:1+events_unbound22:25:270
1297850991714,2cyclictest1277441-21kworker/u8:3+events_unbound19:49:290
1297850991713,3cyclictest1396905-21latency_hist20:45:000
1297850991713,3cyclictest1314116-21gpgv19:24:590
1297850991711,4cyclictest1402872-21diskstats20:50:140
129786099168,7cyclictest0-21swapper/320:35:003
1297860991613,2cyclictest1553788-21irqcore23:15:183
129786099161,13cyclictest1610339-21/usr/sbin/munin00:10:163
129786099161,12cyclictest1579339-21cut23:40:143
129786099161,12cyclictest1512322-21tr22:35:143
129786099161,12cyclictest1439429-21/usr/sbin/munin21:25:143
129786099161,12cyclictest1356203-21sort20:05:183
129786099160,9cyclictest1521997-21chrt22:42:003
129786099160,8cyclictest1568291-21taskset23:26:433
129786099160,8cyclictest1355396-21taskset20:01:233
129786099160,13cyclictest1594586-21grep23:55:143
129786099160,13cyclictest1376334-21latency_hist20:25:013
129786099160,13cyclictest1366268-21/usr/sbin/munin20:15:143
1297856991613,2cyclictest1537409-21kworker/u8:1+flush-259:023:00:172
1297856991613,2cyclictest1443468-21kworker/u8:2+flush-259:021:45:202
1297856991613,2cyclictest1381472-21kworker/u8:0+flush-259:021:10:152
1297856991613,2cyclictest1329900-21kworker/u8:0+events_unbound19:45:172
1297856991613,2cyclictest1260544-21kworker/u8:1+flush-259:019:10:262
1297856991612,3cyclictest1594546-21kworker/u8:0+events_unbound00:10:202
1297856991612,3cyclictest1335133-21kworker/u8:1+events_unbound20:30:172
1297852991614,1cyclictest1637217-21irqcore00:35:181
1297852991613,2cyclictest1568908-21cpuspeed_turbos23:30:141
1297852991613,2cyclictest1381472-21kworker/u8:0+flush-259:021:55:281
1297852991612,3cyclictest1407860-21kworker/u8:4+events_unbound21:40:181
1297850991614,1cyclictest60350irq/132-enp0s29f1:rx-022:55:250
1297850991614,1cyclictest60350irq/132-enp0s29f1:rx-021:40:150
1297850991614,1cyclictest60350irq/132-enp0s29f1:rx-020:45:260
1297850991613,2cyclictest60350irq/132-enp0s29f1:rx-023:30:120
1297850991613,2cyclictest60350irq/132-enp0s29f1:rx-021:04:000
1297850991613,2cyclictest1636794-21apt-get00:35:130
1297850991613,2cyclictest1537409-21kworker/u8:1+flush-259:023:40:250
1297850991613,2cyclictest1407860-21kworker/u8:4+events_unbound21:30:190
1297850991612,3cyclictest1496546-21kworker/u8:0+events_unbound22:35:020
129786099157,5cyclictest1303565-21cat19:15:013
129786099153,10cyclictest1475944-21fschecks_count22:00:173
129786099151,12cyclictest1632503-21sed00:30:173
129786099151,12cyclictest1475507-21turbostat22:00:003
129786099151,12cyclictest1346072-21grep19:55:163
129786099150,13cyclictest1495521-21grep22:15:203
129786099150,13cyclictest1423914-21df21:10:133
129786099150,12cyclictest1538008-21/usr/sbin/munin23:00:113
129786099150,12cyclictest1501974-21apt-get22:25:123
1297856991512,2cyclictest1511692-21kworker/u8:2+flush-259:023:11:392
1297856991512,2cyclictest1443468-21kworker/u8:2+flush-259:021:50:272
1297856991512,2cyclictest1391448-21kworker/u8:3+flush-259:020:45:212
1297856991511,3cyclictest1345413-21kworker/u8:2+flush-259:021:03:202
1297856991511,3cyclictest1297826-21kworker/u8:2+events_unbound19:31:312
129785299157,1cyclictest1386588-21munin-run20:35:001
1297852991513,2cyclictest1381472-21kworker/u8:0+events_unbound20:50:261
1297852991513,1cyclictest0-21swapper/121:10:001
1297852991513,1cyclictest0-21swapper/119:15:001
1297852991512,2cyclictest1423027-21kworker/u8:1+events_unbound21:25:181
1297852991512,2cyclictest1407860-21kworker/u8:4+flush-259:021:39:391
1297852991512,2cyclictest1407860-21kworker/u8:4+flush-259:021:39:391
1297852991512,2cyclictest1345413-21kworker/u8:2+events_unbound20:00:261
1297850991512,2cyclictest60350irq/132-enp0s29f1:rx-019:40:170
1297850991512,2cyclictest1407860-21kworker/u8:4+flush-259:022:30:250
1297850991512,2cyclictest1407860-21kworker/u8:4+events_unbound22:17:550
1297850991512,2cyclictest1345413-21kworker/u8:2+events_unbound20:00:240
1297850991511,3cyclictest1537409-21kworker/u8:1+flush-259:023:55:200
1297850991511,3cyclictest1511692-21kworker/u8:2+events_unbound23:08:300
1297850991511,3cyclictest1511692-21kworker/u8:2+events_unbound00:09:320
1297850991510,4cyclictest0-21swapper/020:25:000
129786099141,11cyclictest1605101-21diskstats00:05:153
129786099140,12cyclictest1592067-21kthreadcore23:50:173
129786099140,12cyclictest1527511-21wc22:50:013
129786099140,12cyclictest1445232-21sed21:30:193
129786099140,12cyclictest1344213-21kthreadcore19:50:193
1297856991411,2cyclictest1567652-21ntp_kernel_err23:25:232
1297856991411,2cyclictest1511692-21kworker/u8:2+events_unbound00:05:232
1297856991411,2cyclictest1501940-21kworker/u8:1+flush-259:022:25:202
1297856991411,2cyclictest1443468-21kworker/u8:2+events_unbound22:15:162
1297856991411,2cyclictest1443468-21kworker/u8:2+events_unbound22:04:292
1297856991411,2cyclictest1407860-21kworker/u8:4+flush-259:022:12:532
1297856991411,2cyclictest1381472-21kworker/u8:0+flush-259:020:40:132
1297852991411,2cyclictest1537409-21kworker/u8:1+events_unbound23:55:251
1297852991411,2cyclictest1511692-21kworker/u8:2+events_unbound00:10:261
1297852991411,2cyclictest1511692-21kworker/u8:2+events_unbound00:03:081
1297850991412,2cyclictest1443468-21kworker/u8:2+flush-259:022:20:230
1297850991412,1cyclictest0-21swapper/022:45:150
1297850991411,2cyclictest1615012-21turbostat00:15:000
1297850991411,2cyclictest1443468-21kworker/u8:2+events_unbound22:10:160
1297850991411,2cyclictest1443468-21kworker/u8:2+events_unbound22:00:240
1297850991411,2cyclictest1335133-21kworker/u8:1+events_unbound20:30:250
1297850991410,3cyclictest1277441-21kworker/u8:3+events_unbound20:05:030
1297856991311,1cyclictest1459725-21latency_hist21:45:002
1297856991310,2cyclictest1591702-21kthreadcore23:50:172
1297856991310,2cyclictest1511692-21kworker/u8:2+flush-259:023:05:162
1297856991310,2cyclictest1496546-21kworker/u8:0+flush-259:022:40:262
1297856991310,2cyclictest1381472-21kworker/u8:0+events_unbound21:25:012
1297856991310,2cyclictest1381472-21kworker/u8:0+events_unbound21:15:302
1297856991310,2cyclictest0-21swapper/222:50:192
129785299138,4cyclictest1329900-21kworker/u8:0+flush-259:019:45:311
129785099139,3cyclictest1511692-21kworker/u8:2+events_unbound00:03:320
1297850991311,1cyclictest60350irq/132-enp0s29f1:rx-020:15:270
129786099120,10cyclictest1297927-21wc19:10:013
129785299129,2cyclictest1407860-21kworker/u8:4+events_unbound21:30:291
129785099129,2cyclictest1537409-21kworker/u8:1+flush-259:023:35:160
129785299118,2cyclictest1297826-21kworker/u8:2+flush-259:019:30:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional