You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 10:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Dec 25, 2025 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
72228994946,2cyclictest351113-21fwupd23:35:211
72228994235,2cyclictest180706-21kworker/u8:3+events_unbound21:40:231
72228994235,2cyclictest180706-21kworker/u8:3+events_unbound21:40:221
72231994033,5cyclictest315704-21latency_hist23:05:012
72228994037,2cyclictest284707-21kworker/u8:5+flush-259:023:00:011
72228993936,2cyclictest206593-21kworker/u8:1+events_unbound21:30:231
72236993832,4cyclictest336742-21latency_hist23:24:593
72231993833,3cyclictest180706-21kworker/u8:3+flush-259:021:25:152
72228993833,3cyclictest118598-21kworker/u8:2+events_unbound20:05:241
72228993734,2cyclictest372992-21kworker/u8:0+flush-259:000:05:241
72228993632,3cyclictest218171-21kworker/u8:2+flush-259:021:40:011
72228993632,3cyclictest218171-21kworker/u8:2+flush-259:021:40:001
72228993531,3cyclictest330525-21kworker/u8:1+flush-259:023:15:231
72228993530,3cyclictest127030-21kworker/u8:1+flush-259:020:50:181
72228993431,2cyclictest144234-21kworker/u8:2+events_unbound20:40:141
72228993430,3cyclictest206593-21kworker/u8:1+flush-259:021:55:251
72228993430,3cyclictest190945-21kworker/u8:0+flush-259:021:20:001
72228993428,4cyclictest409809-21/usr/sbin/munin00:35:121
72231993327,4cyclictest394036-21latency_hist00:20:012
72228993330,2cyclictest284707-21kworker/u8:5+flush-259:023:00:241
72228993330,2cyclictest284707-21kworker/u8:5+events_unbound23:05:241
72228993330,2cyclictest284706-21kworker/u8:4+flush-259:022:50:191
72228993330,2cyclictest206593-21kworker/u8:1+flush-259:022:40:001
72228993329,3cyclictest144234-21kworker/u8:2+events_unbound20:25:171
72236993229,2cyclictest127030-21kworker/u8:1+events_unbound20:31:203
72231993229,2cyclictest144234-21kworker/u8:2+flush-259:020:30:172
72231993226,4cyclictest72138-21latency_hist19:10:002
72228993229,2cyclictest248616-21kworker/u8:4+flush-259:022:20:151
72228993229,2cyclictest190945-21kworker/u8:0+events_unbound21:10:221
72228993229,2cyclictest180706-21kworker/u8:3+events_unbound21:45:161
72228993228,3cyclictest72160-21kworker/u8:4+flush-259:019:30:121
72228993128,2cyclictest72160-21kworker/u8:4+events_unbound19:55:141
72228993128,2cyclictest206593-21kworker/u8:1+events_unbound22:30:191
72236993027,2cyclictest139074-21kworker/u8:0+events_unbound20:19:203
72236993027,2cyclictest103114-21kworker/u8:0+events_unbound19:58:213
72231993026,3cyclictest206593-21kworker/u8:1+flush-259:022:30:202
72231993026,3cyclictest206593-21kworker/u8:1+events_unbound21:35:012
72228993026,3cyclictest351316-21kworker/u8:0+events_unbound23:45:231
72226993025,3cyclictest284707-21kworker/u8:5+flush-259:000:35:000
72226993025,3cyclictest127030-21kworker/u8:1+flush-259:020:10:170
72226993023,5cyclictest248592-21latency_hist22:00:010
72228992926,2cyclictest72160-21kworker/u8:4+events_unbound20:20:211
72228992926,2cyclictest258935-21kworker/u8:0+events_unbound22:15:201
72228992925,3cyclictest362490-21kworker/u8:3+events_unbound23:55:221
72228992925,3cyclictest180706-21kworker/u8:3+events_unbound21:50:241
72236992826,2cyclictest330525-21kworker/u8:1+events_unbound23:30:263
72236992826,2cyclictest243477-21kworker/u8:2+events_unbound22:14:203
72231992825,2cyclictest72160-21kworker/u8:4+flush-259:019:35:162
72231992824,3cyclictest72160-21kworker/u8:4+flush-259:019:20:172
72231992824,3cyclictest72160-21kworker/u8:4+flush-259:019:20:172
72231992822,4cyclictest252-21systemd-journal22:25:012
72228992825,2cyclictest284707-21kworker/u8:5+events_unbound23:50:131
72228992825,2cyclictest243477-21kworker/u8:2+events_unbound22:05:121
72228992825,2cyclictest103114-21kworker/u8:0+flush-259:019:45:191
72226992821,5cyclictest273463-21ntpq22:20:200
72236992724,2cyclictest362490-21kworker/u8:3+events_unbound23:54:213
72236992724,2cyclictest144234-21kworker/u8:2+events_unbound20:57:063
72236992723,3cyclictest127030-21kworker/u8:1+events_unbound20:50:303
72228992723,3cyclictest127030-21kworker/u8:1+events_unbound20:59:591
72226992723,3cyclictest72160-21kworker/u8:4+flush-259:020:20:160
72236992624,2cyclictest72160-21kworker/u8:4+events_unbound19:11:213
72236992623,2cyclictest82531-21kworker/u8:1+events_unbound19:38:203
72231992622,3cyclictest362490-21kworker/u8:3+flush-259:000:00:002
72231992621,3cyclictest252-21systemd-journal19:13:192
72231992620,4cyclictest388879-21latency_hist00:15:012
72228992624,2cyclictest139074-21kworker/u8:0+events_unbound20:15:141
72228992623,2cyclictest72160-21kworker/u8:4+events_unbound19:10:131
72228992623,2cyclictest243477-21kworker/u8:2+flush-259:022:00:161
72228992622,3cyclictest383340-21kworker/u8:2+events_unbound00:25:181
72228992622,3cyclictest190945-21kworker/u8:0+events_unbound21:25:021
72236992522,2cyclictest362490-21kworker/u8:3+events_unbound23:58:213
72236992522,2cyclictest218171-21kworker/u8:2+events_unbound21:31:213
72231992522,2cyclictest383340-21kworker/u8:2+flush-259:000:25:202
72231992521,3cyclictest127030-21kworker/u8:1+flush-259:021:00:252
72228992522,2cyclictest72160-21kworker/u8:4+events_unbound19:50:131
72228992522,2cyclictest144234-21kworker/u8:2+flush-259:020:30:201
72228992522,2cyclictest118598-21kworker/u8:2+flush-259:020:00:261
72226992520,4cyclictest368029-21apt-get23:55:120
72226992520,4cyclictest284707-21kworker/u8:5+flush-259:023:35:210
72226992518,5cyclictest310960-21diskstats23:00:130
72236992422,2cyclictest87892-21kworker/u8:2+events_unbound19:31:473
72236992422,2cyclictest34239-21kworker/u8:0+events_unbound19:21:483
72236992422,2cyclictest34239-21kworker/u8:0+events_unbound19:21:473
72236992422,2cyclictest284707-21kworker/u8:5+events_unbound00:34:563
72236992422,2cyclictest284707-21kworker/u8:5+events_unbound00:28:393
72236992422,2cyclictest284706-21kworker/u8:4+events_unbound22:47:273
72236992422,2cyclictest180706-21kworker/u8:3+events_unbound21:10:263
72236992421,2cyclictest284706-21kworker/u8:4+events_unbound22:50:293
72236992421,2cyclictest118598-21kworker/u8:2+events_unbound20:09:213
72231992421,2cyclictest180706-21kworker/u8:3+flush-259:021:55:012
72231992420,3cyclictest127030-21kworker/u8:1+flush-259:020:45:162
72231992420,3cyclictest127030-21kworker/u8:1+flush-259:020:05:002
72228992420,3cyclictest190945-21kworker/u8:0+flush-259:021:20:231
72228992419,3cyclictest252-21systemd-journal19:25:191
72226992417,5cyclictest118551-21latency_hist19:55:010
72236992320,2cyclictest351316-21kworker/u8:0+events_unbound23:43:223
72236992320,2cyclictest34239-21kworker/u8:0+events_unbound19:19:193
72236992320,2cyclictest34239-21kworker/u8:0+events_unbound19:19:193
72228992321,2cyclictest299307-21kworker/u8:0+events_unbound23:14:201
72228992320,2cyclictest72160-21kworker/u8:4+events_unbound19:15:211
72228992320,2cyclictest72160-21kworker/u8:4+events_unbound19:15:201
72228992320,2cyclictest284707-21kworker/u8:5+flush-259:023:30:141
72226992317,4cyclictest243477-21kworker/u8:2+flush-259:022:15:160
72226992316,5cyclictest196160-21latency_hist21:10:000
7222699230,21cyclictest1-21systemd21:29:200
72236992221,1cyclictest299307-21kworker/u8:0+events_unbound22:56:213
72236992221,1cyclictest144234-21kworker/u8:2+events_unbound20:21:593
72236992220,2cyclictest206593-21kworker/u8:1+events_unbound22:18:213
72236992220,2cyclictest127030-21kworker/u8:1+events_unbound20:13:203
72236992219,2cyclictest144234-21kworker/u8:2+events_unbound20:49:203
72236992219,2cyclictest144234-21kworker/u8:2+events_unbound20:35:273
72226992218,2cyclictest351316-21kworker/u8:0+flush-259:023:40:160
72226992216,4cyclictest190945-21kworker/u8:0+flush-259:021:24:590
72226992215,5cyclictest185852-21latency_hist21:00:000
72236992119,2cyclictest284706-21kworker/u8:4+events_unbound23:25:013
72236992119,2cyclictest284706-21kworker/u8:4+events_unbound22:40:013
72236992119,2cyclictest180706-21kworker/u8:3+events_unbound21:07:083
72236992118,2cyclictest180706-21kworker/u8:3+events_unbound21:36:213
72236992118,2cyclictest180706-21kworker/u8:3+events_unbound21:36:203
72231992117,3cyclictest372992-21kworker/u8:0+flush-259:000:00:262
72231992115,4cyclictest252-21systemd-journal20:22:002
72228992118,2cyclictest372992-21kworker/u8:0+events_unbound00:16:271
72228992118,2cyclictest284707-21kworker/u8:5+events_unbound23:25:171
72228992118,2cyclictest127030-21kworker/u8:1+flush-259:020:45:161
72226992115,4cyclictest284707-21kworker/u8:5+flush-259:000:35:150
72226992114,5cyclictest367770-21latency_hist23:55:010
72226992114,5cyclictest320831-21latency_hist23:10:000
72236992017,2cyclictest72160-21kworker/u8:4+events_unbound19:51:593
72236992017,2cyclictest372992-21kworker/u8:0+events_unbound00:06:083
72236992017,2cyclictest351316-21kworker/u8:0+events_unbound23:46:203
72236992017,2cyclictest330525-21kworker/u8:1+events_unbound00:01:193
72236992017,2cyclictest299307-21kworker/u8:0+events_unbound23:09:063
72236992017,2cyclictest180706-21kworker/u8:3+events_unbound21:20:023
72231992017,2cyclictest284706-21kworker/u8:4+flush-259:023:20:212
72231992016,3cyclictest72160-21kworker/u8:4+events_unbound19:40:242
72231992016,3cyclictest351316-21kworker/u8:0+flush-259:023:45:202
72231992016,3cyclictest284706-21kworker/u8:4+events_unbound22:45:242
72228992017,2cyclictest82531-21kworker/u8:1+flush-259:019:35:251
72226992016,3cyclictest72160-21kworker/u8:4+flush-259:019:20:200
72226992016,3cyclictest72160-21kworker/u8:4+flush-259:019:20:190
72226992016,3cyclictest252-21systemd-journal22:40:010
72236991917,2cyclictest72160-21kworker/u8:4+events_unbound20:25:043
72236991917,2cyclictest413922-21kworker/u8:0+events_unbound00:36:213
72236991916,2cyclictest72160-21kworker/u8:4+events_unbound19:40:393
72236991916,2cyclictest284707-21kworker/u8:5+events_unbound00:18:363
72236991916,2cyclictest243477-21kworker/u8:2+events_unbound21:59:563
72236991916,2cyclictest127030-21kworker/u8:1+events_unbound21:03:213
72236991915,3cyclictest284706-21kworker/u8:4+events_unbound22:36:543
72228991916,2cyclictest330525-21kworker/u8:1+flush-259:000:03:211
72228991916,2cyclictest243477-21kworker/u8:2+flush-259:022:12:201
72228991916,2cyclictest127030-21kworker/u8:1+flush-259:021:04:201
72228991915,3cyclictest72160-21kworker/u8:4+flush-259:020:35:141
72226991913,4cyclictest372992-21kworker/u8:0+flush-259:000:05:170
72236991816,2cyclictest82531-21kworker/u8:1+events_unbound19:28:483
72236991816,1cyclictest2971-21cron20:45:003
72236991815,2cyclictest389121-21kworker/u8:1+events_unbound00:22:183
72236991815,2cyclictest330525-21kworker/u8:1+events_unbound23:16:153
72236991815,2cyclictest254424-21irqcore22:05:163
72236991815,2cyclictest206593-21kworker/u8:1+events_unbound22:30:033
72236991815,2cyclictest206593-21kworker/u8:1+events_unbound22:24:143
72236991815,2cyclictest118598-21kworker/u8:2+events_unbound20:02:143
72231991816,1cyclictest144234-21kworker/u8:2+events_unbound20:25:232
72231991815,2cyclictest82531-21kworker/u8:1+flush-259:019:25:212
72231991815,2cyclictest258935-21kworker/u8:0+flush-259:022:15:242
72231991814,3cyclictest372992-21kworker/u8:0+events_unbound00:05:202
72231991814,3cyclictest330525-21kworker/u8:1+flush-259:023:35:152
72231991814,3cyclictest299307-21kworker/u8:0+events_unbound23:15:002
72231991814,3cyclictest243477-21kworker/u8:2+flush-259:022:05:202
72231991813,3cyclictest331543-21awk23:20:002
72231991812,4cyclictest289006-21ntpq22:35:212
72228991815,2cyclictest180706-21kworker/u8:3+flush-259:021:05:151
72228991814,3cyclictest284707-21kworker/u8:5+events_unbound22:45:161
72228991812,4cyclictest252-21systemd-journal00:10:011
72226991813,3cyclictest330525-21kworker/u8:1+flush-259:023:30:270
72226991811,5cyclictest299265-21ntpq22:45:200
7223699175,3cyclictest50-21ksoftirqd/323:35:223
72236991715,2cyclictest284707-21kworker/u8:5+events_unbound00:12:333
72236991715,2cyclictest190945-21kworker/u8:0+events_unbound21:26:323
72236991715,2cyclictest103114-21kworker/u8:0+events_unbound19:46:383
72236991714,2cyclictest284707-21kworker/u8:5+events_unbound23:01:543
72236991714,2cyclictest273963-21kworker/u8:3+events_unbound22:29:223
72236991714,2cyclictest206593-21kworker/u8:1+events_unbound21:45:353
72236991714,2cyclictest180706-21kworker/u8:3+events_unbound21:41:003
72236991714,2cyclictest180706-21kworker/u8:3+events_unbound21:41:003
72236991710,5cyclictest252-21systemd-journal21:15:183
72231991713,3cyclictest389121-21kworker/u8:1+flush-259:000:20:202
72231991713,3cyclictest351316-21kworker/u8:0+events_unbound23:40:142
72231991713,3cyclictest243477-21kworker/u8:2+flush-259:021:55:182
72231991713,3cyclictest190945-21kworker/u8:0+events_unbound21:20:232
72228991714,2cyclictest72160-21kworker/u8:4+flush-259:019:40:201
72228991714,2cyclictest206593-21kworker/u8:1+events_unbound22:25:131
72226991714,2cyclictest61150irq/133-enp0s29f1:tx-023:10:170
72226991714,2cyclictest60350irq/132-enp0s29f1:rx-020:50:140
72226991714,2cyclictest330525-21kworker/u8:1+flush-259:000:05:000
72226991713,3cyclictest60350irq/132-enp0s29f1:rx-021:15:130
72226991713,3cyclictest190945-21kworker/u8:0+events_unbound21:10:210
72236991613,2cyclictest243477-21kworker/u8:2+events_unbound22:01:183
72236991613,2cyclictest180706-21kworker/u8:3+events_unbound21:52:253
72236991611,1cyclictest191rcu_preempt23:10:203
72231991613,2cyclictest72160-21kworker/u8:4+flush-259:019:48:492
72231991613,2cyclictest248616-21kworker/u8:4+events_unbound22:10:212
72231991612,3cyclictest284707-21kworker/u8:5+events_unbound23:25:192
72231991612,3cyclictest206593-21kworker/u8:1+events_unbound22:25:202
72231991611,4cyclictest72160-21kworker/u8:4+flush-259:020:35:202
72231991611,3cyclictest180706-21kworker/u8:3+flush-259:021:05:212
7223199160,14cyclictest509-21rs:main2
72228991613,2cyclictest383340-21kworker/u8:2+flush-259:000:20:251
72228991613,2cyclictest383340-21kworker/u8:2+events_unbound00:30:121
72226991614,1cyclictest60350irq/132-enp0s29f1:rx-022:40:340
72226991614,1cyclictest60350irq/132-enp0s29f1:rx-000:20:120
72226991614,1cyclictest60350irq/132-enp0s29f1:rx-000:15:140
72226991613,2cyclictest61150irq/133-enp0s29f1:tx-023:15:220
72226991613,2cyclictest60350irq/132-enp0s29f1:rx-021:31:020
72226991613,2cyclictest60350irq/132-enp0s29f1:rx-020:40:340
72226991612,3cyclictest72160-21kworker/u8:4+flush-259:019:15:000
72226991612,3cyclictest60350irq/132-enp0s29f1:rx-021:00:210
72231991511,3cyclictest362490-21kworker/u8:3+events_unbound23:50:252
72231991511,3cyclictest284706-21kworker/u8:4+events_unbound22:50:132
72231991511,2cyclictest82518-21sh19:20:012
72231991511,2cyclictest82518-21sh19:20:002
72228991512,2cyclictest34239-21kworker/u8:0+events_unbound19:21:021
72228991512,2cyclictest34239-21kworker/u8:0+events_unbound19:21:021
72228991512,2cyclictest284706-21kworker/u8:4+flush-259:023:20:261
7222699159,4cyclictest98192-21apt-get19:35:120
7222699159,4cyclictest383340-21kworker/u8:2+flush-259:000:25:260
7222699158,5cyclictest331964-21diskstats23:20:140
72226991513,2cyclictest61150irq/133-enp0s29f1:tx-021:40:250
72226991513,2cyclictest61150irq/133-enp0s29f1:tx-021:40:250
72226991513,2cyclictest60350irq/132-enp0s29f1:rx-023:47:390
72226991513,2cyclictest60350irq/132-enp0s29f1:rx-020:35:180
72226991513,1cyclictest60350irq/132-enp0s29f1:rx-022:10:150
72226991513,1cyclictest60350irq/132-enp0s29f1:rx-022:00:270
72226991513,1cyclictest60350irq/132-enp0s29f1:rx-020:05:270
72226991513,1cyclictest60350irq/132-enp0s29f1:rx-019:55:060
72226991512,2cyclictest60350irq/132-enp0s29f1:rx-020:18:250
72226991512,2cyclictest60350irq/132-enp0s29f1:rx-020:04:190
72226991512,2cyclictest60350irq/132-enp0s29f1:rx-019:45:170
72226991510,3cyclictest144234-21kworker/u8:2+flush-259:020:25:250
72231991412,1cyclictest0-21swapper/220:15:002
72231991411,2cyclictest228649-21irqcore21:40:182
72231991411,2cyclictest228649-21irqcore21:40:172
72231991411,2cyclictest199139-21kthreadcore21:10:182
72231991410,3cyclictest118598-21kworker/u8:2+flush-259:020:05:212
72231991410,2cyclictest0-21swapper/219:55:002
72228991412,2cyclictest284707-21kworker/u8:5+events_unbound23:40:281
72228991411,2cyclictest284707-21kworker/u8:5+events_unbound22:42:531
7222699148,4cyclictest340996-21ntpq23:25:230
72226991413,1cyclictest60350irq/132-enp0s29f1:rx-022:29:080
72226991413,1cyclictest60350irq/132-enp0s29f1:rx-021:35:200
72226991413,1cyclictest60350irq/132-enp0s29f1:rx-021:35:190
72226991413,1cyclictest60350irq/132-enp0s29f1:rx-020:32:490
72226991412,1cyclictest60350irq/132-enp0s29f1:rx-020:45:180
72226991412,1cyclictest60350irq/132-enp0s29f1:rx-019:40:170
72226991411,2cyclictest60350irq/132-enp0s29f1:rx-022:06:190
72226991411,2cyclictest60350irq/132-enp0s29f1:rx-021:45:130
72226991411,2cyclictest60350irq/132-enp0s29f1:rx-019:15:220
72226991411,2cyclictest60350irq/132-enp0s29f1:rx-019:15:220
7223699130,3cyclictest72293-21sed19:10:003
7223199139,3cyclictest284707-21kworker/u8:5+flush-259:022:40:252
7223199139,3cyclictest180706-21kworker/u8:3+flush-259:021:35:222
7223199139,3cyclictest180706-21kworker/u8:3+flush-259:021:35:222
7223199139,3cyclictest127030-21kworker/u8:1+flush-259:020:50:222
72231991311,2cyclictest389121-21kworker/u8:1+flush-259:000:39:502
72231991310,2cyclictest72220-21cyclictest22:05:002
72231991310,2cyclictest181967-21kthreadcore20:55:162
7222699137,5cyclictest305662-21apt-get22:55:130
72226991311,2cyclictest60350irq/132-enp0s29f1:rx-000:12:210
72226991311,1cyclictest60350irq/132-enp0s29f1:rx-022:31:530
7223199129,2cyclictest405083-21irqcore00:30:182
7223199129,2cyclictest330525-21kworker/u8:1+events_unbound23:30:202
7223199128,3cyclictest180706-21kworker/u8:3+flush-259:021:45:182
7223199128,3cyclictest144234-21kworker/u8:2+flush-259:020:42:022
7222699129,2cyclictest61150irq/133-enp0s29f1:tx-019:30:160
72226991210,1cyclictest60350irq/132-enp0s29f1:rx-021:50:110
7223199119,1cyclictest0-21swapper/221:20:002
7223199118,2cyclictest93449-21sed19:30:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional