You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-08 - 00:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Wed Jan 07, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3474251994138,2cyclictest3748752-21kworker/u8:0+flush-259:012:10:211
3474254993731,4cyclictest3593605-21latency_hist09:05:012
3474251993528,5cyclictest3624502-21latency_hist09:35:011
3474259993431,2cyclictest3748752-21kworker/u8:0+flush-259:011:45:003
3474251993329,3cyclictest3780273-21kworker/u8:1+flush-259:012:35:221
3474251993229,2cyclictest3562880-21kworker/u8:4+flush-259:008:35:211
3474251993128,2cyclictest3603894-21kworker/u8:2+flush-259:009:55:251
3474251993125,4cyclictest3474164-21latency_hist07:10:011
3474251993026,3cyclictest3603894-21kworker/u8:2+events_unbound11:00:211
3474251993026,3cyclictest3603894-21kworker/u8:2+events_unbound09:45:121
3474251993024,4cyclictest3671466-21/usr/sbin/munin10:20:161
3474251992925,3cyclictest3686968-21kworker/u8:4+flush-259:010:40:171
3474251992823,4cyclictest3598970-21/usr/sbin/munin09:10:161
3474259992620,4cyclictest3666508-21diskstats10:15:133
3474254992623,2cyclictest3733836-21kworker/u8:4+flush-259:011:35:222
3474251992623,2cyclictest3420858-21kworker/u8:1+flush-259:009:20:201
3474251992622,3cyclictest3748752-21kworker/u8:0+events_unbound11:50:171
3474251992622,2cyclictest3578343-21kworker/u8:3+events_unbound08:50:181
3474249992623,3cyclictest3603894-21kworker/u8:2+events_unbound10:10:010
3474251992522,2cyclictest3420858-21kworker/u8:1+flush-259:008:30:201
3474251992521,3cyclictest3748752-21kworker/u8:0+flush-259:012:00:211
3474251992520,4cyclictest3733182-21apt-get11:20:011
3474251992518,5cyclictest3550998-21ntpq08:20:221
3474251992421,2cyclictest3712573-21kworker/u8:3+events_unbound11:45:141
3474251992421,2cyclictest3603894-21kworker/u8:2+flush-259:010:00:151
3474251992420,3cyclictest3603894-21kworker/u8:2+flush-259:009:40:151
3474249992420,3cyclictest3515912-21kworker/u8:3+flush-259:008:10:170
3474249992419,3cyclictest3712550-21latency_hist11:00:000
3474249992418,4cyclictest252-21systemd-journal11:10:380
3474259992319,3cyclictest3733208-21gpgv11:20:003
3474259992318,4cyclictest3588440-21kworker/u8:0+flush-259:009:00:163
3474254992319,3cyclictest3686968-21kworker/u8:4+flush-259:010:40:122
3474251992318,3cyclictest3676677-21apt-get10:25:121
3474259992215,5cyclictest3609029-21awk09:20:013
3474251992219,2cyclictest3733836-21kworker/u8:4+flush-259:012:15:201
3474251992219,2cyclictest3603894-21kworker/u8:2+flush-259:009:25:211
3474251992218,3cyclictest3464030-21kworker/u8:0+events_unbound07:25:281
3474249992216,4cyclictest3562648-21latency_hist08:35:000
3474259992118,2cyclictest3697081-21sh10:45:003
3474254992117,3cyclictest3801307-21gpgv12:25:002
3474254992115,4cyclictest3515891-21latency_hist07:50:012
3474251992118,2cyclictest3603894-21kworker/u8:2+flush-259:010:50:211
3474251992117,3cyclictest3733836-21kworker/u8:4+events_unbound11:25:171
3474251992117,3cyclictest3603894-21kworker/u8:2+flush-259:011:15:001
3474251992117,3cyclictest3484157-21kworker/u8:2+events_unbound08:00:241
3474259992017,2cyclictest3521294-21apt-get07:55:123
3474259992015,4cyclictest3780742-21apt-get12:05:113
3474259992014,4cyclictest3420858-21kworker/u8:1+flush-259:007:45:173
3474259992014,4cyclictest252-21systemd-journal08:45:003
3474259992014,4cyclictest252-21systemd-journal08:45:003
3474254992014,4cyclictest3785641-21latency_hist12:10:002
3474251992017,2cyclictest3588440-21kworker/u8:0+events_unbound09:00:271
3474251992016,3cyclictest3603894-21kworker/u8:2+events_unbound10:15:221
3474251992016,3cyclictest3495535-21/usr/sbin/munin07:30:151
3474249992017,2cyclictest507-21in:imuxsock11:20:010
3474249992015,4cyclictest252-21systemd-journal10:39:590
3474249992014,4cyclictest3515912-21kworker/u8:3+flush-259:008:25:000
3474259991913,4cyclictest252-21systemd-journal09:25:393
3474254991916,1cyclictest0-21swapper/212:20:012
3474254991913,4cyclictest3609032-21latency_hist09:20:012
3474251991916,2cyclictest3780273-21kworker/u8:1+flush-259:012:20:261
3474251991916,2cyclictest3666104-21latency_hist10:15:001
3474251991915,2cyclictest3603894-21kworker/u8:2+events_unbound10:35:171
3474249991914,4cyclictest3546783-21latency_hist08:20:000
3474259991812,4cyclictest3770204-21latency_hist11:55:013
3474259991812,4cyclictest3645321-21apt-get09:55:133
3474259991812,4cyclictest252-21systemd-journal12:25:023
3474259991811,5cyclictest3811027-21ntpq12:30:203
347425499185,7cyclictest3645060-21munin-run09:55:002
3474254991815,2cyclictest3765005-21latency_hist11:50:012
3474254991814,3cyclictest3484157-21kworker/u8:2+flush-259:008:00:002
3474251991815,2cyclictest0-21swapper/107:15:001
3474251991814,3cyclictest3588440-21kworker/u8:0+events_unbound09:15:261
3474251991814,2cyclictest3655407-21kworker/u8:3+events_unbound10:05:211
3474249991815,2cyclictest211rcuc/011:10:000
3474249991813,3cyclictest60350irq/132-enp0s29f1:rx-011:25:140
3474249991813,3cyclictest3546818-21kworker/u8:0+flush-259:008:45:230
3474249991813,3cyclictest3515897-21turbostat07:50:000
3474259991714,2cyclictest3464030-21kworker/u8:0+flush-259:007:25:213
3474254991715,1cyclictest0-21swapper/207:30:012
3474254991714,2cyclictest3630291-21irqcore09:40:172
3474254991713,3cyclictest3603894-21kworker/u8:2+flush-259:011:00:242
3474254991712,3cyclictest3583540-21apt-get08:55:122
3474251991714,2cyclictest3806985-21kworker/u8:0+events_unbound12:30:141
3474251991714,2cyclictest3578343-21kworker/u8:3+flush-259:009:05:151
3474251991714,2cyclictest3420858-21kworker/u8:1+flush-259:007:45:171
3474251991712,3cyclictest3572968-21kworker/u8:2+events_unbound08:55:181
3474251991710,5cyclictest3753397-21ntpq11:35:191
3474249991714,2cyclictest60350irq/132-enp0s29f1:rx-011:55:210
3474249991713,3cyclictest3733836-21kworker/u8:4+flush-259:011:20:220
3474249991713,3cyclictest3489926-21/usr/sbin/munin07:25:140
3474249991713,2cyclictest61150irq/133-enp0s29f1:tx-011:45:110
3474249991712,3cyclictest3812179-21apt-get12:35:120
3474249991712,3cyclictest3603894-21kworker/u8:2+flush-259:010:00:170
3474249991711,4cyclictest252-21systemd-journal09:06:390
3474259991612,3cyclictest3655407-21kworker/u8:3+flush-259:010:30:173
3474259991612,2cyclictest3578377-21apt-get08:50:113
3474259991610,4cyclictest3479645-21apt-get07:15:123
3474254991613,2cyclictest3621892-21kthreadcore09:30:172
3474254991612,3cyclictest500-21systemd-logind08:30:002
3474254991612,3cyclictest3655407-21kworker/u8:3+flush-259:010:35:002
3474254991612,3cyclictest3420858-21kworker/u8:1+flush-259:008:40:202
3474254991612,3cyclictest3420858-21kworker/u8:1+flush-259:008:40:202
3474254991612,3cyclictest252-21systemd-journal12:40:002
3474254991611,3cyclictest3536734-21apt-get08:10:132
3474254991610,4cyclictest252-21systemd-journal11:35:012
3474251991613,2cyclictest3748752-21kworker/u8:0+flush-259:011:40:171
3474251991613,2cyclictest3544178-21kthreadcore08:15:201
3474251991613,2cyclictest3484157-21kworker/u8:2+events_unbound07:43:461
3474251991612,3cyclictest3733836-21kworker/u8:4+events_unbound11:20:151
3474251991612,3cyclictest3420858-21kworker/u8:1+events_unbound08:28:401
3474249991614,1cyclictest60350irq/132-enp0s29f1:rx-012:02:570
3474249991614,1cyclictest60350irq/132-enp0s29f1:rx-008:00:160
3474249991614,1cyclictest60350irq/132-enp0s29f1:rx-007:41:270
3474249991613,2cyclictest61150irq/133-enp0s29f1:tx-007:20:390
3474249991613,2cyclictest60350irq/132-enp0s29f1:rx-012:29:090
3474249991613,2cyclictest60350irq/132-enp0s29f1:rx-010:25:180
3474249991613,2cyclictest60350irq/132-enp0s29f1:rx-010:24:100
3474249991613,2cyclictest60350irq/132-enp0s29f1:rx-008:40:130
3474249991613,2cyclictest60350irq/132-enp0s29f1:rx-008:40:120
3474249991612,2cyclictest3501145-21irqcore07:35:180
347425499159,4cyclictest3665878-21gpgv10:15:012
347425499151,12cyclictest3790839-21awk12:15:012
3474251991512,2cyclictest3801404-21kworker/u8:2+flush-259:012:25:201
3474251991512,2cyclictest3712573-21kworker/u8:3+events_unbound11:05:241
3474249991513,2cyclictest60350irq/132-enp0s29f1:rx-009:01:440
3474249991513,1cyclictest60350irq/132-enp0s29f1:rx-012:20:150
3474249991513,1cyclictest60350irq/132-enp0s29f1:rx-012:09:220
3474249991513,1cyclictest60350irq/132-enp0s29f1:rx-009:10:170
3474249991513,1cyclictest60350irq/132-enp0s29f1:rx-008:55:210
3474249991513,1cyclictest60350irq/132-enp0s29f1:rx-008:25:160
3474249991513,1cyclictest60350irq/132-enp0s29f1:rx-007:55:120
3474249991513,1cyclictest60350irq/132-enp0s29f1:rx-007:50:200
3474249991512,2cyclictest60350irq/132-enp0s29f1:rx-011:30:140
3474249991512,2cyclictest60350irq/132-enp0s29f1:rx-009:54:090
3474249991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:100
3474249991512,2cyclictest60350irq/132-enp0s29f1:rx-009:31:210
3474249991512,2cyclictest60350irq/132-enp0s29f1:rx-009:25:250
3474249991511,3cyclictest3748752-21kworker/u8:0+flush-259:012:20:000
347425999149,3cyclictest3780273-21kworker/u8:1+flush-259:012:35:163
347425999149,3cyclictest3603894-21kworker/u8:2+flush-259:009:35:013
3474259991411,2cyclictest3780067-21unixbench_singl12:00:253
3474259991411,2cyclictest3712560-21idleruntime-cro11:00:003
3474259991411,2cyclictest3681580-21cron10:30:003
347425499142,6cyclictest3759444-21awk11:45:002
3474254991411,2cyclictest3578343-21kworker/u8:3+flush-259:009:05:202
3474254991410,3cyclictest3712573-21kworker/u8:3+flush-259:011:10:302
3474254991410,3cyclictest3706997-21kworker/u8:0+flush-259:010:55:262
347425199149,3cyclictest3546818-21kworker/u8:0+events_unbound08:45:221
3474251991411,2cyclictest3733836-21kworker/u8:4+flush-259:011:55:311
3474251991411,2cyclictest3484157-21kworker/u8:2+flush-259:007:55:201
3474251991410,3cyclictest3464030-21kworker/u8:0+events_unbound07:15:231
3474251991410,2cyclictest0-21swapper/110:45:241
347424999148,4cyclictest3495385-21base6407:30:010
3474249991413,1cyclictest60350irq/132-enp0s29f1:rx-009:40:250
3474249991412,2cyclictest60350irq/132-enp0s29f1:rx-008:51:340
3474249991412,1cyclictest60350irq/132-enp0s29f1:rx-011:50:190
3474249991412,1cyclictest60350irq/132-enp0s29f1:rx-010:49:340
3474249991412,1cyclictest60350irq/132-enp0s29f1:rx-010:15:200
3474249991412,1cyclictest60350irq/132-enp0s29f1:rx-010:10:150
3474249991412,1cyclictest60350irq/132-enp0s29f1:rx-009:15:150
3474249991412,1cyclictest60350irq/132-enp0s29f1:rx-007:17:150
3474249991411,2cyclictest60350irq/132-enp0s29f1:rx-009:35:130
3474249991411,2cyclictest60350irq/132-enp0s29f1:rx-009:20:250
3474249991411,2cyclictest60350irq/132-enp0s29f1:rx-007:12:210
3474249991410,3cyclictest3806985-21kworker/u8:0+flush-259:012:30:190
347425999139,3cyclictest3484157-21kworker/u8:2+flush-259:008:10:253
3474259991311,1cyclictest0-21swapper/308:30:013
3474259991310,2cyclictest3546299-21unixbench_multi08:15:263
3474259991310,2cyclictest0-21swapper/309:15:003
3474254991310,2cyclictest3809314-21kthreadcore12:30:162
3474254991310,1cyclictest0-21swapper/211:20:012
347425199139,2cyclictest0-21swapper/108:45:001
347425199139,2cyclictest0-21swapper/108:45:001
3474251991310,2cyclictest3484157-21kworker/u8:2+flush-259:008:05:181
3474251991310,1cyclictest0-21swapper/107:50:201
347425199130,5cyclictest3489694-21munin-plugin-st07:25:001
3474249991310,2cyclictest60350irq/132-enp0s29f1:rx-011:42:110
3474249991310,2cyclictest60350irq/132-enp0s29f1:rx-008:05:240
347425999128,3cyclictest3733836-21kworker/u8:4+flush-259:011:25:253
347425999128,3cyclictest3484157-21kworker/u8:2+flush-259:007:30:183
347425999126,4cyclictest3801378-21cp12:25:003
347425999126,4cyclictest3629622-21latency_hist09:40:013
3474259991211,0cyclictest1414006-21kworker/3:3+events_power_efficient08:35:163
347425499129,1cyclictest0-21swapper/207:35:182
347425499128,3cyclictest3635156-21cpuspeed_turbos09:45:112
347425499127,3cyclictest252-21systemd-journal11:50:022
347425499127,3cyclictest252-21systemd-journal11:25:012
347425499126,4cyclictest3562914-21apt-get08:35:132
3474254991210,1cyclictest0-21swapper/212:25:232
347425199129,2cyclictest3748752-21kworker/u8:0+events_unbound12:05:161
347425199128,3cyclictest3603894-21kworker/u8:2+events_unbound10:56:201
347425199127,3cyclictest3712573-21kworker/u8:3+flush-259:011:30:061
347424999129,2cyclictest60350irq/132-enp0s29f1:rx-011:35:170
347424999129,2cyclictest60350irq/132-enp0s29f1:rx-011:00:220
347424999129,2cyclictest60350irq/132-enp0s29f1:rx-008:35:120
3474249991210,1cyclictest60350irq/132-enp0s29f1:rx-010:41:200
347425999119,1cyclictest0-21swapper/312:15:133
347425999118,2cyclictest3754244-21sh11:40:013
347425999118,2cyclictest0-21swapper/310:35:123
347425999118,2cyclictest0-21swapper/309:55:013
347425999117,2cyclictest17650irq/131-nvme0q409:25:003
347425999116,3cyclictest3733846-21/usr/sbin/munin11:20:213
347425999116,3cyclictest252-21systemd-journal07:11:393
347425999111,8cyclictest3505582-21turbostat.cron07:40:003
347425499118,2cyclictest3609392-21cpuspeed_turbos09:20:122
347425499118,2cyclictest3582264-21needreboot08:50:192
347425499118,2cyclictest3548055-21kthreadcore08:20:182
347425499118,2cyclictest3545845-21ntp_kernel_err08:15:222
347425499118,2cyclictest2971-21cron10:30:002
347425499118,2cyclictest0-21swapper/209:10:152
347425499118,1cyclictest0-21swapper/210:35:182
347425499117,3cyclictest3712573-21kworker/u8:3+flush-259:011:05:172
347425499116,3cyclictest513-21gdbus09:30:012
347425499110,6cyclictest3500463-21munin-run07:35:002
347425499110,3cyclictest36702872nfs_client10:15:192
347425199117,3cyclictest3603894-21kworker/u8:2+flush-259:010:30:251
347424999119,1cyclictest60350irq/132-enp0s29f1:rx-010:52:230
347425999108,1cyclictest0-21swapper/307:20:113
347425999107,2cyclictest3635156-21cpuspeed_turbos09:45:113
347425999104,4cyclictest3707402-21awk10:55:003
347425499108,1cyclictest0-21swapper/209:55:142
347425499107,2cyclictest3624829-21cpuspeed_turbos09:35:132
347425499106,3cyclictest3603894-21kworker/u8:2+flush-259:010:00:242
347425499106,3cyclictest3489549-21sshd07:23:392
34742549996,2cyclictest3478936-21unixbench_singl07:10:262
34742549995,3cyclictest3557058-21apt-get08:30:022
34742549992,5cyclictest3706421-21grep10:50:202
34742549990,2cyclictest3775355-21awk12:00:012
34742549986,1cyclictest0-21swapper/207:15:152
34742549983,3cyclictest3535751-21/usr/sbin/munin08:05:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional