You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 10:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri May 15, 2026 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3298869995535,14cyclictest3549390-21/usr/sbin/munin23:10:171
3298867995348,3cyclictest252-21systemd-journal20:42:090
3298875994841,5cyclictest3371898-21cp20:20:013
3298869994437,5cyclictest3409083-21latency_hist20:55:001
3298867994440,3cyclictest3538560-21gpgv23:00:000
3298869994337,4cyclictest252-21systemd-journal22:44:441
3298867994238,3cyclictest3334940-21kworker/u8:1+flush-259:019:45:160
3298875994136,3cyclictest252-21systemd-journal00:31:413
3298872994136,3cyclictest3475662-21apt-get21:55:202
3298875994036,3cyclictest3456080-21kworker/u8:0+flush-259:021:45:263
3298875994035,3cyclictest252-21systemd-journal22:20:433
3298872994037,2cyclictest3456080-21kworker/u8:0+flush-259:022:10:002
3298875993932,5cyclictest3569756-21latency_hist23:30:013
3298872993934,3cyclictest252-21systemd-journal20:27:092
3298867993932,5cyclictest3334396-21ntpq19:40:200
3298867993731,5cyclictest3605738-21latency_hist00:05:000
3298872993631,4cyclictest3563732-21ntpq23:20:212
3298875993530,4cyclictest3554289-21latency_hist23:15:003
3298869993530,3cyclictest3371996-21gpgv20:20:001
3298867993528,5cyclictest3606161-21diskstats00:05:160
3298875993430,3cyclictest3502358-21kworker/u8:2+flush-259:022:35:243
3298869993428,4cyclictest252-21systemd-journal22:52:331
3298875993326,5cyclictest3398783-21latency_hist20:45:003
3298869993320,11cyclictest3350334-21ntpq19:55:211
3298867993326,5cyclictest3393655-21latency_hist20:40:000
3298872993127,3cyclictest3502358-21kworker/u8:2+flush-259:022:25:002
3298872993127,3cyclictest3335587-21apt-get19:45:122
3298869993125,5cyclictest3570025-21apt-get23:30:121
3298867993127,2cyclictest211rcuc/021:35:000
3298872993027,2cyclictest3456080-21kworker/u8:0+flush-259:022:05:002
3298867993024,4cyclictest3502595-21apt-get22:25:090
3298875992925,3cyclictest3626323-21apt-get00:25:003
3298875992925,3cyclictest3334940-21kworker/u8:1+events_unbound20:20:163
3298867992925,3cyclictest3424574-21kworker/u8:3+flush-259:022:15:440
3298875992824,3cyclictest3408727-21kworker/u8:3+flush-259:020:55:243
3298872992822,5cyclictest3444230-21ntpq21:25:222
3298872992822,5cyclictest3444230-21ntpq21:25:222
3298872992822,4cyclictest3497169-21latency_hist22:20:012
3298869992822,4cyclictest3487151-21apt-get22:10:121
3298867992824,3cyclictest3512640-21kworker/u8:5+flush-259:023:00:180
3298867992824,2cyclictest3340486-21kworker/u8:3+events_unbound20:10:170
3298867992823,4cyclictest3388121-21apt-get20:35:010
3298867992822,4cyclictest3314674-21diskstats19:25:130
3298875992724,2cyclictest3507737-21apt-get22:30:123
3298872992722,3cyclictest252-21systemd-journal21:01:432
3298872992721,4cyclictest3460132-21ntpq21:40:202
3298872992721,4cyclictest3377372-21latency_hist20:25:012
3298872992721,4cyclictest3303973-21latency_hist19:15:012
3298869992720,5cyclictest3626323-21apt-get00:25:021
3298867992723,3cyclictest3512640-21kworker/u8:5+events_unbound23:17:380
3298867992722,4cyclictest3393418-21kworker/u8:0+flush-259:020:50:140
3298867992722,3cyclictest3351451-21apt-get20:00:110
3298867992721,4cyclictest252-21systemd-journal21:37:430
3298875992624,1cyclictest3424574-21kworker/u8:3+events_unbound22:15:213
3298872992622,3cyclictest3424574-21kworker/u8:3+flush-259:023:10:252
3298872992621,3cyclictest3329787-21gpgv19:39:592
3298867992621,3cyclictest3502358-21kworker/u8:2+events_unbound22:35:220
3298867992620,4cyclictest3319429-21kworker/u8:0+flush-259:019:55:210
3298875992520,3cyclictest252-21systemd-journal23:35:433
3298872992521,3cyclictest3340486-21kworker/u8:3+events_unbound20:10:192
3298872992520,3cyclictest3455642-21cat21:40:002
3298872992519,4cyclictest252-21systemd-journal19:19:432
3298869992518,5cyclictest3335322-21latency_hist19:45:011
3298867992520,3cyclictest3569804-21kworker/u8:2+events_unbound23:55:240
3298867992519,4cyclictest3329761-21apt-get19:40:000
3298872992421,2cyclictest3456080-21kworker/u8:0+flush-259:021:50:242
3298872992420,3cyclictest3512640-21kworker/u8:5+flush-259:022:55:262
3298869992419,4cyclictest3409347-21apt-get20:55:111
3298867992420,3cyclictest3522991-21kworker/u8:0+events_unbound22:45:230
3298867992420,3cyclictest3502358-21kworker/u8:2+flush-259:022:30:210
3298867992417,5cyclictest3372479-21apt-get20:20:100
3298875992320,2cyclictest3585409-21kworker/u8:1+events_unbound00:00:003
3298875992320,2cyclictest3456080-21kworker/u8:0+flush-259:022:10:213
3298875992319,3cyclictest509-21rs:main3
3298872992319,3cyclictest509-21rs:main2
3298869992320,2cyclictest3249333-21kworker/u8:2+flush-259:019:30:181
3298869992317,4cyclictest3419649-21apt-get21:05:111
3298867992319,3cyclictest3366349-21kworker/u8:0+events_unbound20:15:160
3298867992318,3cyclictest3382979-21kworker/u8:2+events_unbound21:20:220
3298875992219,2cyclictest3424574-21kworker/u8:3+flush-259:022:30:003
3298875992216,4cyclictest3314255-21latency_hist19:25:013
3298872992217,4cyclictest3554528-21/usr/sbin/munin23:15:222
3298869992219,2cyclictest3626348-21gpgv00:25:001
3298867992218,3cyclictest3605344-21kworker/u8:3+events_unbound00:10:220
3298867992218,3cyclictest3414289-21kworker/u8:0+events_unbound21:02:160
3298867992218,3cyclictest3382979-21kworker/u8:2+events_unbound21:40:180
3298872992118,2cyclictest3334940-21kworker/u8:1+flush-259:020:45:132
3298872992117,3cyclictest3605344-21kworker/u8:3+flush-259:000:10:002
3298872992117,3cyclictest3366349-21kworker/u8:0+flush-259:020:15:242
3298872992117,3cyclictest3249333-21kworker/u8:2+flush-259:019:30:212
3298869992117,3cyclictest3616007-21kworker/u8:4+flush-259:000:15:221
3298867992116,4cyclictest3309317-21kworker/u8:4+flush-259:019:20:170
3298875992017,2cyclictest3456080-21kworker/u8:0+events_unbound22:05:223
3298875992016,3cyclictest3424574-21kworker/u8:3+flush-259:023:40:173
3298875992016,3cyclictest3424574-21kworker/u8:3+flush-259:023:40:163
3298875992016,3cyclictest3424574-21kworker/u8:3+flush-259:022:00:003
3298872992018,1cyclictest3466204-21turbostat.cron21:50:002
3298872992017,2cyclictest3424574-21kworker/u8:3+flush-259:023:25:182
3298872992017,2cyclictest3350804-21kworker/u8:2+flush-259:020:00:242
3298872992016,3cyclictest3625962-21kworker/u8:0+flush-259:000:35:202
3298872992015,4cyclictest3450256-21latency_hist21:35:002
3298872992014,4cyclictest3528152-21latency_hist22:50:012
3298867992016,3cyclictest3461244-21kworker/u8:4+flush-259:021:55:160
3298867992015,4cyclictest3424574-21kworker/u8:3+events_unbound21:25:150
3298867992015,4cyclictest3424574-21kworker/u8:3+events_unbound21:25:150
3298867992015,3cyclictest3424574-21kworker/u8:3+flush-259:023:35:180
3298867992014,4cyclictest509-21rs:main0
3298867992014,4cyclictest3512640-21kworker/u8:5+flush-259:022:50:210
3298867992014,4cyclictest3424574-21kworker/u8:3+flush-259:021:15:210
3298875991915,3cyclictest3340464-21latency_hist19:50:013
3298875991914,3cyclictest252-21systemd-journal20:35:013
3298872991915,3cyclictest3408727-21kworker/u8:3+flush-259:020:55:202
3298872991914,4cyclictest3344727-21ntpq19:50:202
3298869991914,4cyclictest252-21systemd-journal00:40:001
3298869991913,4cyclictest3324790-21/usr/sbin/munin19:35:191
3298867991916,2cyclictest3424574-21kworker/u8:3+events_unbound21:10:240
3298867991915,3cyclictest3585409-21kworker/u8:1+events_unbound23:50:240
3298867991914,3cyclictest3424574-21kworker/u8:3+events_unbound23:40:150
3298867991914,3cyclictest3424574-21kworker/u8:3+events_unbound23:40:150
3298875991815,2cyclictest3249333-21kworker/u8:2+flush-259:019:40:213
3298872991815,2cyclictest3424574-21kworker/u8:3+flush-259:023:05:002
3298872991814,3cyclictest3439969-21latency_hist21:25:002
3298869991813,3cyclictest3507468-21awk22:30:001
3298867991815,2cyclictest3641456-21ntpq00:35:240
3298867991813,3cyclictest252-21systemd-journal23:10:430
3298875991714,2cyclictest3345688-21apt-key19:55:003
3298875991714,1cyclictest0-21swapper/320:30:013
3298875991712,4cyclictest252-21systemd-journal21:40:013
3298869991713,3cyclictest3424574-21kworker/u8:3+flush-259:023:40:211
3298869991713,3cyclictest3424574-21kworker/u8:3+flush-259:023:40:211
3298869991713,3cyclictest3424574-21kworker/u8:3+flush-259:022:00:251
3298869991711,4cyclictest3589402-21ntpq23:45:211
3298869991711,4cyclictest252-21systemd-journal19:45:421
3298867991713,3cyclictest3456080-21kworker/u8:0+events_unbound21:49:440
3298867991713,2cyclictest3424574-21kworker/u8:3+events_unbound23:25:180
3298867991712,4cyclictest3424574-21kworker/u8:3+events_unbound23:21:450
3298867991712,4cyclictest3249333-21kworker/u8:2+flush-259:019:30:150
3298867991712,3cyclictest3554334-21kworker/u8:0+flush-259:023:30:160
3298867991711,4cyclictest3456080-21kworker/u8:0+events_unbound22:12:260
3298867991710,5cyclictest3403944-21latency_hist20:50:000
3298875991613,2cyclictest3627423-21sed00:25:173
3298875991611,4cyclictest252-21systemd-journal00:20:013
3298872991614,1cyclictest0-21swapper/223:55:012
3298872991613,2cyclictest3456080-21kworker/u8:0+flush-259:022:10:162
3298872991613,2cyclictest3424574-21kworker/u8:3+flush-259:021:15:262
3298869991613,1cyclictest0-21swapper/119:55:011
3298867991614,1cyclictest60350irq/132-enp0s29f1:rx-000:25:170
3298867991612,2cyclictest3486888-21cron22:10:000
3298867991611,4cyclictest3585409-21kworker/u8:1+flush-259:023:45:200
3298875991512,2cyclictest3522991-21kworker/u8:0+flush-259:022:45:253
3298875991510,3cyclictest252-21systemd-journal21:18:433
3298872991512,2cyclictest3424574-21kworker/u8:3+flush-259:023:45:012
3298872991512,2cyclictest3424574-21kworker/u8:3+flush-259:023:45:002
3298872991512,2cyclictest3387217-21ntp_kernel_pll_20:30:202
3298872991512,2cyclictest3249333-21kworker/u8:2+flush-259:019:30:002
3298872991510,3cyclictest252-21systemd-journal23:33:432
329886999159,4cyclictest3371797-21apt-get20:20:011
329886999158,4cyclictest252-21systemd-journal19:21:421
3298869991512,2cyclictest3449402-21ntp_kernel_pll_21:30:231
3298869991511,3cyclictest3605344-21kworker/u8:3+flush-259:000:10:161
3298869991511,3cyclictest3366624-21sh20:15:011
3298869991510,3cyclictest3523207-21/usr/sbin/munin22:45:151
329886799158,5cyclictest3304205-21/usr/sbin/munin19:15:130
3298867991513,2cyclictest60350irq/132-enp0s29f1:rx-022:42:430
3298867991513,1cyclictest60350irq/132-enp0s29f1:rx-022:00:230
3298867991512,2cyclictest60350irq/132-enp0s29f1:rx-019:10:120
3298867991512,2cyclictest3512640-21kworker/u8:5+events_unbound23:05:230
3298867991511,3cyclictest3616007-21kworker/u8:4+events_unbound00:20:200
329887599148,4cyclictest3346473-21diskstats19:55:143
3298875991412,1cyclictest0-21swapper/323:45:183
3298875991411,2cyclictest3533682-21cpuspeed_turbos22:55:133
3298875991410,3cyclictest3585409-21kworker/u8:1+flush-259:000:05:253
3298875991410,3cyclictest3456080-21kworker/u8:0+flush-259:021:50:273
3298872991412,1cyclictest0-21swapper/222:35:002
3298872991411,2cyclictest61750irq/145-enp0s29f1:tx-622:40:212
3298872991411,2cyclictest3569804-21kworker/u8:2+events_unbound23:55:202
3298872991410,3cyclictest3340486-21kworker/u8:3+flush-259:020:35:232
3298872991410,3cyclictest3334940-21kworker/u8:1+events_unbound20:40:222
3298869991412,1cyclictest0-21swapper/123:25:181
3298869991411,2cyclictest3471374-21cron21:55:001
3298869991411,2cyclictest0-21swapper/122:59:591
3298869991410,3cyclictest3585409-21kworker/u8:1+flush-259:023:55:251
3298869991410,3cyclictest3575138-21/usr/sbin/munin23:35:161
3298867991413,1cyclictest60350irq/132-enp0s29f1:rx-021:53:250
3298867991412,2cyclictest60350irq/132-enp0s29f1:rx-020:59:260
3298867991412,1cyclictest60350irq/132-enp0s29f1:rx-022:20:100
3298867991412,1cyclictest60350irq/132-enp0s29f1:rx-021:07:170
3298867991412,1cyclictest60350irq/132-enp0s29f1:rx-000:30:260
3298867991411,2cyclictest60350irq/132-enp0s29f1:rx-020:27:000
3298875991311,1cyclictest3548325-21ntp_offset23:05:223
3298875991311,1cyclictest0-21swapper/320:15:013
3298875991310,2cyclictest3637471-21cpuspeed_turbos00:35:153
3298875991310,2cyclictest3518512-21irqcore22:40:163
3298875991310,2cyclictest3424574-21kworker/u8:3+events_unbound21:10:223
3298875991310,2cyclictest3356765-21fschecks_count20:05:153
3298872991310,2cyclictest3427101-21kthreadcore21:10:192
3298872991310,2cyclictest3424574-21kworker/u8:3+flush-259:023:35:252
329886999139,3cyclictest500-21systemd-logind23:10:001
329886999139,3cyclictest3559789-21cpuspeed_turbos23:20:131
329886999139,3cyclictest3340486-21kworker/u8:3+flush-259:020:30:241
329886999137,4cyclictest252-21systemd-journal21:45:431
329886999136,5cyclictest3559459-21latency_hist23:20:001
3298869991310,2cyclictest3512950-21cpuspeed_turbos22:35:141
3298867991311,1cyclictest60350irq/132-enp0s29f1:rx-019:50:120
3298867991310,2cyclictest60350irq/132-enp0s29f1:rx-020:05:140
329887599129,2cyclictest3393418-21kworker/u8:0+flush-259:020:50:163
329887599129,2cyclictest3391082-21tr20:35:183
329887599128,3cyclictest3293926-21kworker/u8:0+flush-259:019:15:493
3298875991210,1cyclictest0-21swapper/322:05:003
329887299128,3cyclictest3616007-21kworker/u8:4+flush-259:000:20:362
329887299121,9cyclictest3615989-21latency_hist00:15:002
329886999129,2cyclictest3450700-21fschecks_count21:35:151
329886999129,2cyclictest3402934-21needreboot20:45:191
329886999129,1cyclictest0-21swapper/121:55:201
329886999128,2cyclictest0-21swapper/120:40:001
329886999127,3cyclictest3435097-21apt-get21:20:121
329887599119,2cyclictest3424574-21kworker/u8:3+flush-259:023:20:233
329887599119,1cyclictest0-21swapper/321:40:003
329887599118,2cyclictest3590988-21irqcore23:50:173
329887599117,3cyclictest252-21systemd-journal19:35:003
329887599115,4cyclictest252-21systemd-journal00:15:003
329887299119,1cyclictest0-21swapper/200:25:182
329887299117,2cyclictest0-21swapper/200:05:012
329886999116,3cyclictest3511717-21sed22:30:211
329886999116,3cyclictest252-21systemd-journal20:42:441
329887599108,1cyclictest832-21runrttasks19:35:243
329887599108,1cyclictest3419724-21cpuspeed_turbos21:05:123
329887599108,1cyclictest0-21swapper/321:00:153
329887599108,1cyclictest0-21swapper/319:10:243
329887599107,2cyclictest3352466-21kthreadcore20:00:173
329887599106,3cyclictest3436322-21cut21:20:183
329887599100,2cyclictest3449448-21/usr/sbin/munin21:30:243
329887299106,3cyclictest3512640-21kworker/u8:5+flush-259:023:07:402
329887299106,3cyclictest3393418-21kworker/u8:0+events_unbound20:50:222
329887299106,2cyclictest3532469-21ntpq22:50:222
329887299106,2cyclictest3314255-21latency_hist19:25:002
329887299104,4cyclictest490-21dbus-daemon23:50:002
329887299100,2cyclictest3637141-21latency_hist00:35:012
32988759996,2cyclictest3554906-21if_enp0s30f423:15:153
32988729996,2cyclictest401ktimers/222:35:222
32988729990,7cyclictest3424534-21cron21:10:002
32988759985,2cyclictest491ktimers/323:00:023
32988759984,3cyclictest491ktimers/323:30:133
32988759984,3cyclictest3314801-21fw_packets19:25:143
32988729986,1cyclictest0-21swapper/220:05:152
32988729984,3cyclictest3346127-21apt-config19:55:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional