You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-09 - 07:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Nov 09, 2025 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
514740993529,4cyclictest519861-21kworker/u8:1+flush-259:019:25:191
514736993327,4cyclictest608363-21latency_hist20:40:010
514742993228,3cyclictest706395-21kworker/u8:0+flush-259:022:20:012
514740993023,5cyclictest659993-21latency_hist21:30:011
514736993027,2cyclictest732834-21apt-get22:40:110
514736993026,3cyclictest520101-21apt-get19:15:120
514740992824,3cyclictest743507-21kworker/u8:1+flush-259:023:45:181
514740992721,4cyclictest3000063-21rs:main1
514746992623,2cyclictest530374-21kworker/u8:4+events_unbound21:20:003
514746992620,4cyclictest2698871-21systemd-journal21:30:013
514742992622,3cyclictest836740-21/usr/sbin/munin00:20:212
514740992621,3cyclictest624117-21/usr/sbin/munin20:55:201
514740992621,3cyclictest3000063-21rs:main1
514736992623,2cyclictest685741-21kworker/u8:1+flush-259:022:25:260
514736992623,2cyclictest685741-21kworker/u8:1+flush-259:022:25:250
514740992521,3cyclictest743508-21kworker/u8:4+events_unbound23:15:161
514746992421,2cyclictest743507-21kworker/u8:1+flush-259:022:55:003
514746992421,2cyclictest743507-21kworker/u8:1+events_unbound23:45:223
514740992421,2cyclictest492-21dbus-daemon20:30:011
514740992418,4cyclictest2698871-21systemd-journal23:25:011
514746992316,5cyclictest602781-21latency_hist20:35:013
514740992319,3cyclictest795077-21kworker/u8:0+flush-259:023:50:191
514740992319,3cyclictest624109-21kworker/u8:1+events_unbound21:05:241
514740992318,4cyclictest2698871-21systemd-journal23:00:281
514736992317,4cyclictest519841-21latency_hist19:15:010
514746992219,2cyclictest629062-21sh21:00:013
514746992218,3cyclictest753622-21kworker/u8:2+flush-259:023:45:013
514740992219,2cyclictest670313-21kworker/u8:3+events_unbound21:50:181
514740992217,4cyclictest2698871-21systemd-journal20:41:291
514736992219,2cyclictest685741-21kworker/u8:1+flush-259:022:05:250
514736992217,4cyclictest753622-21kworker/u8:2+flush-259:023:40:180
514736992216,5cyclictest633299-21ntpq21:00:200
514746992119,1cyclictest0-21swapper/322:45:013
514742992118,2cyclictest603432-21kworker/u8:2+flush-259:020:45:252
514740992117,3cyclictest685979-21apt-get21:55:091
514746992017,2cyclictest670313-21kworker/u8:3+events_unbound22:37:293
514746992017,2cyclictest519861-21kworker/u8:1+events_unbound19:25:263
514746992015,2cyclictest0-21swapper/319:39:593
514742992017,2cyclictest540654-21kworker/u8:2+flush-259:019:50:272
514740992017,2cyclictest743507-21kworker/u8:1+events_unbound00:20:141
514740992016,3cyclictest530374-21kworker/u8:4+flush-259:020:10:151
514740992016,3cyclictest530374-21kworker/u8:4+flush-259:020:10:151
514736992014,4cyclictest2698871-21systemd-journal19:55:000
514746991916,2cyclictest499689-21kworker/u8:3+events_unbound19:55:163
514746991915,3cyclictest685741-21kworker/u8:1+events_unbound22:21:283
514746991915,3cyclictest592277-21kworker/u8:3+events_unbound20:50:243
514746991915,3cyclictest530374-21kworker/u8:4+events_unbound20:10:193
514746991915,3cyclictest530374-21kworker/u8:4+events_unbound20:10:193
514746991915,3cyclictest472063-21kworker/u8:2+events_unbound19:20:183
514746991914,4cyclictest706615-21/usr/sbin/munin22:15:183
514742991916,2cyclictest499689-21kworker/u8:3+flush-259:019:40:172
514742991915,3cyclictest685741-21kworker/u8:1+flush-259:022:25:212
514742991915,3cyclictest685741-21kworker/u8:1+flush-259:022:25:202
514740991917,2cyclictest795077-21kworker/u8:0+flush-259:000:00:241
514740991916,2cyclictest634221-21kworker/u8:0+events_unbound21:40:281
514740991914,4cyclictest634221-21kworker/u8:0+flush-259:021:20:161
514740991914,4cyclictest602773-21apt-get20:35:011
514740991914,3cyclictest586909-21latency_hist20:20:011
514740991914,3cyclictest586909-21latency_hist20:20:001
514740991912,5cyclictest795085-21/usr/sbin/munin23:40:201
514736991915,3cyclictest826213-21kworker/u8:3+flush-259:000:34:590
514736991914,3cyclictest2698871-21systemd-journal21:10:010
514736991913,5cyclictest549897-21ntpq19:40:220
514746991815,2cyclictest540654-21kworker/u8:2+events_unbound19:50:283
514746991815,2cyclictest530374-21kworker/u8:4+events_unbound20:24:563
514746991815,2cyclictest499689-21kworker/u8:3+events_unbound19:45:143
514746991814,3cyclictest753622-21kworker/u8:2+flush-259:023:30:153
514746991811,5cyclictest762965-21ntpq23:05:223
514746991811,5cyclictest684868-21ntpq21:50:233
514742991812,4cyclictest769057-21latency_hist23:15:012
514740991816,1cyclictest602864-21apt-key20:35:001
514740991815,2cyclictest566272-21kworker/u8:1+flush-259:020:05:281
514740991814,3cyclictest685741-21kworker/u8:1+events_unbound22:10:121
514740991814,3cyclictest540654-21kworker/u8:2+events_unbound19:50:301
514740991813,3cyclictest603432-21kworker/u8:2+flush-259:020:45:181
514746991714,2cyclictest654853-21idleruntime-cro21:25:003
514746991713,3cyclictest795077-21kworker/u8:0+flush-259:023:50:223
514746991713,3cyclictest701494-21apt-get22:10:113
514746991713,3cyclictest624109-21kworker/u8:1+events_unbound21:09:563
514746991713,3cyclictest499689-21kworker/u8:3+events_unbound20:00:203
514746991713,3cyclictest472063-21kworker/u8:2+events_unbound19:16:303
514746991710,5cyclictest835641-21ntpq00:15:223
514742991713,3cyclictest795077-21kworker/u8:0+flush-259:023:50:282
51474099176,9cyclictest831391-21munin-run00:15:001
514740991714,2cyclictest743508-21kworker/u8:4+events_unbound23:25:221
514740991714,2cyclictest644759-21kworker/u8:2+events_unbound21:15:281
514740991713,3cyclictest738133-21gpgv22:45:011
514740991713,3cyclictest670313-21kworker/u8:3+events_unbound22:20:191
514736991714,2cyclictest634221-21kworker/u8:0+flush-259:021:45:170
514736991713,3cyclictest634221-21kworker/u8:0+flush-259:021:25:180
514736991712,3cyclictest60650irq/133-enp0s29f1:rx-021:35:130
514746991613,2cyclictest685741-21kworker/u8:1+flush-259:022:30:263
514746991612,3cyclictest826213-21kworker/u8:3+events_unbound00:28:513
514746991612,3cyclictest815502-21kworker/u8:2+events_unbound00:03:413
514746991612,3cyclictest795077-21kworker/u8:0+events_unbound00:35:153
514746991612,2cyclictest743508-21kworker/u8:4+events_unbound23:20:243
514746991610,4cyclictest2698871-21systemd-journal20:20:013
514746991610,4cyclictest2698871-21systemd-journal20:20:003
514742991613,2cyclictest743507-21kworker/u8:1+flush-259:023:05:172
514740991613,2cyclictest670313-21kworker/u8:3+flush-259:022:35:161
514740991612,3cyclictest743508-21kworker/u8:4+flush-259:023:55:241
514740991610,4cyclictest851930-21latency_hist00:35:011
514736991614,1cyclictest60650irq/133-enp0s29f1:rx-023:30:260
514736991614,1cyclictest0-21swapper/023:10:010
514736991613,2cyclictest60650irq/133-enp0s29f1:rx-022:00:200
514736991613,2cyclictest60650irq/133-enp0s29f1:rx-019:55:230
514736991612,3cyclictest670313-21kworker/u8:3+events_unbound22:20:170
514736991610,4cyclictest2698871-21systemd-journal23:15:000
514746991512,2cyclictest670313-21kworker/u8:3+events_unbound22:00:233
514746991512,2cyclictest472063-21kworker/u8:2+flush-259:019:10:193
514746991511,3cyclictest743508-21kworker/u8:4+flush-259:023:15:013
514746991511,3cyclictest706395-21kworker/u8:0+flush-259:022:45:283
514746991511,3cyclictest624109-21kworker/u8:1+events_unbound21:10:213
514746991511,3cyclictest608366-21idleruntime-cro20:40:003
514746991511,2cyclictest0-21swapper/323:15:193
514742991512,2cyclictest722955-21irqcore22:30:162
514742991512,2cyclictest634221-21kworker/u8:0+flush-259:021:40:182
514742991511,3cyclictest530374-21kworker/u8:4+flush-259:020:20:012
514742991511,3cyclictest530374-21kworker/u8:4+flush-259:020:20:002
51474099159,4cyclictest821294-21/usr/sbin/munin00:05:141
514740991512,2cyclictest753622-21kworker/u8:2+events_unbound23:30:171
514740991511,3cyclictest685741-21kworker/u8:1+flush-259:022:28:341
514740991511,3cyclictest685741-21kworker/u8:1+flush-259:022:28:341
514740991511,3cyclictest624109-21kworker/u8:1+events_unbound21:10:141
514736991514,1cyclictest60650irq/133-enp0s29f1:rx-020:55:190
514736991514,1cyclictest60650irq/133-enp0s29f1:rx-020:03:160
514736991513,1cyclictest60650irq/133-enp0s29f1:rx-022:45:150
514736991513,1cyclictest60650irq/133-enp0s29f1:rx-022:10:400
514736991513,1cyclictest60650irq/133-enp0s29f1:rx-021:40:260
514736991513,1cyclictest60650irq/133-enp0s29f1:rx-020:15:270
514736991513,1cyclictest60650irq/133-enp0s29f1:rx-020:15:260
514736991512,2cyclictest60650irq/133-enp0s29f1:rx-023:50:240
514736991512,2cyclictest60650irq/133-enp0s29f1:rx-021:10:070
514736991512,2cyclictest60650irq/133-enp0s29f1:rx-020:42:480
514736991512,2cyclictest60650irq/133-enp0s29f1:rx-020:10:010
514736991511,3cyclictest2698871-21systemd-journal20:33:280
514746991411,2cyclictest592277-21kworker/u8:3+flush-259:020:40:203
514746991411,2cyclictest540654-21kworker/u8:2+events_unbound19:40:213
514746991410,3cyclictest0-21swapper/300:30:203
514742991411,2cyclictest520169-21cpuspeed_turbos19:15:142
51474299141,11cyclictest1-21systemd00:39:282
51474099148,4cyclictest555020-21ntpq19:45:211
514740991411,2cyclictest685741-21kworker/u8:1+events_unbound22:30:221
514740991411,2cyclictest540654-21kworker/u8:2+flush-259:019:40:261
514740991411,2cyclictest472063-21kworker/u8:2+events_unbound19:20:301
514740991410,3cyclictest795077-21kworker/u8:0+events_unbound00:15:221
514740991410,3cyclictest472063-21kworker/u8:2+events_unbound19:15:151
51473699149,3cyclictest830480-21ntpq00:10:250
514736991413,1cyclictest60650irq/133-enp0s29f1:rx-023:04:060
514736991413,1cyclictest60650irq/133-enp0s29f1:rx-022:59:240
514736991413,1cyclictest60650irq/133-enp0s29f1:rx-021:34:500
514736991413,1cyclictest60650irq/133-enp0s29f1:rx-021:16:250
514736991413,1cyclictest60650irq/133-enp0s29f1:rx-020:25:300
514736991413,1cyclictest60650irq/133-enp0s29f1:rx-000:25:290
514736991412,2cyclictest61450irq/134-enp0s29f1:tx-023:15:200
514736991412,1cyclictest60650irq/133-enp0s29f1:rx-022:50:220
514736991412,1cyclictest60650irq/133-enp0s29f1:rx-022:30:250
514736991412,1cyclictest60650irq/133-enp0s29f1:rx-021:50:060
514736991412,1cyclictest60650irq/133-enp0s29f1:rx-019:32:420
514736991412,1cyclictest587586-21irqcore20:20:180
514736991411,2cyclictest814616-21ntp_kernel_pll_23:55:230
514736991411,2cyclictest685741-21kworker/u8:1+flush-259:021:55:140
514736991411,2cyclictest61450irq/134-enp0s29f1:tx-019:25:260
514736991410,2cyclictest0-21swapper/020:55:010
51474699139,3cyclictest826213-21kworker/u8:3+flush-259:000:25:003
51474699139,3cyclictest586928-21kworker/u8:1+events_unbound20:25:163
51474699139,3cyclictest530374-21kworker/u8:4+flush-259:020:06:333
51474699138,4cyclictest795077-21kworker/u8:0+events_unbound00:05:043
514746991310,2cyclictest743508-21kworker/u8:4+flush-259:023:59:593
514746991310,2cyclictest0-21swapper/322:25:183
514746991310,2cyclictest0-21swapper/322:25:173
51474299139,3cyclictest670313-21kworker/u8:3+flush-259:022:20:172
514742991311,1cyclictest0-21swapper/223:05:002
514742991311,1cyclictest0-21swapper/222:50:142
514742991311,1cyclictest0-21swapper/220:55:002
514742991310,2cyclictest686055-21cpuspeed_turbos21:55:112
514742991310,2cyclictest662549-21kthreadcore21:30:182
514742991310,2cyclictest624109-21kworker/u8:1+flush-259:021:10:182
514742991310,2cyclictest597874-21/usr/sbin/munin20:30:202
514742991310,2cyclictest540419-21cron19:35:002
514742991310,2cyclictest508-21tpm2-abrmd21:40:002
514742991310,2cyclictest0-21swapper/220:00:012
51474099138,4cyclictest530374-21kworker/u8:4+flush-259:020:01:331
51473699138,4cyclictest472063-21kworker/u8:2+flush-259:019:20:190
514736991311,2cyclictest60650irq/133-enp0s29f1:rx-021:22:250
514736991311,1cyclictest60650irq/133-enp0s29f1:rx-020:45:130
514736991311,1cyclictest60650irq/133-enp0s29f1:rx-020:10:470
514736991311,1cyclictest60650irq/133-enp0s29f1:rx-020:10:470
514736991311,1cyclictest60650irq/133-enp0s29f1:rx-000:35:170
514736991311,1cyclictest60650irq/133-enp0s29f1:rx-000:00:440
514736991310,2cyclictest60650irq/133-enp0s29f1:rx-000:20:520
51474699129,2cyclictest634221-21kworker/u8:0+flush-259:021:45:283
51474699129,2cyclictest629749-21irqcore21:00:153
51474699127,3cyclictest753622-21kworker/u8:2+events_unbound23:39:583
51474299129,2cyclictest753622-21kworker/u8:2+flush-259:023:30:262
51474299129,2cyclictest691346-21fschecks_time22:00:162
51474299129,2cyclictest0-21swapper/220:20:182
51474299127,3cyclictest670313-21kworker/u8:3+flush-259:021:50:182
51474299122,5cyclictest613537-21munin-run20:45:012
514742991210,1cyclictest541104-21irqcore19:35:212
514742991210,1cyclictest0-21swapper/222:45:132
51474099129,2cyclictest743508-21kworker/u8:4+flush-259:022:55:161
51474099129,2cyclictest743508-21kworker/u8:4+flush-259:022:50:231
51474099129,2cyclictest581005-21kworker/u8:2+flush-259:020:22:331
51474099129,2cyclictest499689-21kworker/u8:3+events_unbound19:55:231
51474099128,3cyclictest706395-21kworker/u8:0+flush-259:022:45:241
51474099128,3cyclictest696029-21cron22:05:001
514740991210,1cyclictest0-21swapper/123:15:001
514736991211,1cyclictest60650irq/133-enp0s29f1:rx-023:35:140
514736991210,1cyclictest60650irq/133-enp0s29f1:rx-022:38:270
51474699118,2cyclictest634221-21kworker/u8:0+flush-259:021:30:213
51474699117,3cyclictest743507-21kworker/u8:1+events_unbound23:00:203
51474699116,4cyclictest613792-21apt-get20:45:123
51474699114,5cyclictest492-21dbus-daemon22:10:013
51474299119,1cyclictest631621-21kthreadcore21:00:172
51474299119,1cyclictest0-21swapper/222:14:592
51474299118,2cyclictest784067-21unixbench_singl23:25:242
51474299117,3cyclictest603387-21base6420:35:012
51474299117,3cyclictest514960-21apt-get19:10:122
51474099119,1cyclictest0-21swapper/123:40:011
51474099118,2cyclictest795077-21kworker/u8:0+events_unbound00:35:181
51474099117,3cyclictest743507-21kworker/u8:1+flush-259:023:05:211
51474099116,4cyclictest2698871-21systemd-journal20:55:011
51474099111,2cyclictest665142-21latency_hist21:35:011
51473699119,1cyclictest60650irq/133-enp0s29f1:rx-023:45:220
51473699119,1cyclictest60650irq/133-enp0s29f1:rx-000:09:290
51473699118,2cyclictest211rcuc/000:15:190
51473699117,2cyclictest0-21swapper/023:30:010
51474299108,1cyclictest0-21swapper/200:05:142
51474299107,2cyclictest832669-21kthreadcore00:15:182
51474299107,2cyclictest17250irq/130-nvme0q300:30:002
51474299106,3cyclictest732411-21sshd22:38:192
51474299106,3cyclictest571678-21/usr/sbin/munin20:05:182
51474099108,1cyclictest0-21swapper/121:35:151
51474099107,2cyclictest634221-21kworker/u8:0+events_unbound21:45:231
51474099106,3cyclictest519861-21kworker/u8:1+flush-259:019:33:331
5147429997,1cyclictest0-21swapper/220:00:132
5147409997,1cyclictest0-21swapper/119:10:201
5147409994,4cyclictest631188-21cut21:00:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional