You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 18:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Dec 28, 2025 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
767382994235,4cyclictest1042029-21latency_hist11:35:010
767395994036,3cyclictest793620-21kworker/u8:5+events_unbound07:35:233
767395993430,3cyclictest1010902-21kworker/u8:3+flush-259:011:05:003
767395993428,4cyclictest813812-21latency_hist07:55:003
767395993330,2cyclictest828917-21kworker/u8:2+flush-259:008:50:213
767395993229,2cyclictest881329-21kworker/u8:0+flush-259:009:20:003
767395993228,2cyclictest865840-21kworker/u8:1+events_unbound09:04:593
767382993227,3cyclictest1036523-21apt-get11:26:200
767395993129,2cyclictest762207-21kworker/u8:3+events_unbound07:55:233
767395993128,2cyclictest990557-21kworker/u8:3+flush-259:010:45:243
767395993128,2cyclictest772500-21kworker/u8:0+flush-259:007:30:213
767395993127,3cyclictest907663-21kworker/u8:2+flush-259:010:25:113
767395993127,3cyclictest772500-21kworker/u8:0+flush-259:008:05:233
767395993127,2cyclictest1078543-21kworker/u8:0+events_unbound12:30:163
767395993127,2cyclictest1010876-21kworker/u8:2+flush-259:011:15:003
767387993125,4cyclictest902706-21diskstats09:20:151
767395993026,3cyclictest881329-21kworker/u8:0+flush-259:009:20:213
767395993026,3cyclictest1010902-21kworker/u8:3+flush-259:012:10:163
767395993024,4cyclictest808694-21latency_hist07:50:023
767387993026,3cyclictest990365-21kworker/u8:0+flush-259:010:55:001
767382993026,3cyclictest1010902-21kworker/u8:3+flush-259:011:40:220
767395992925,3cyclictest969918-21kworker/u8:4+flush-259:010:40:263
767395992825,2cyclictest907663-21kworker/u8:2+events_unbound09:35:233
767395992824,3cyclictest1010902-21kworker/u8:3+events_unbound12:35:233
767387992823,3cyclictest866063-21/usr/sbin/munin08:45:231
767382992825,2cyclictest1010902-21kworker/u8:3+events_unbound11:50:010
767395992724,2cyclictest907663-21kworker/u8:2+flush-259:010:15:173
767395992724,2cyclictest907663-21kworker/u8:2+events_unbound09:50:143
767395992724,2cyclictest1010902-21kworker/u8:3+flush-259:011:15:153
767387992721,4cyclictest252-21systemd-journal08:10:001
767382992723,3cyclictest881561-21apt-get09:00:130
767382992720,5cyclictest1082818-21ntpq12:10:230
767395992623,2cyclictest907663-21kworker/u8:2+flush-259:009:30:223
767395992623,2cyclictest865840-21kworker/u8:1+events_unbound08:55:143
767387992622,3cyclictest969918-21kworker/u8:4+flush-259:010:35:191
767395992522,2cyclictest850149-21kworker/u8:0+events_unbound08:35:193
767395992522,2cyclictest840305-21kworker/u8:3+flush-259:009:05:163
767395992522,2cyclictest828917-21kworker/u8:2+flush-259:008:40:133
767395992521,3cyclictest2939414-21kworker/u8:1+events_unbound07:10:133
767395992521,3cyclictest1010902-21kworker/u8:3+events_unbound11:50:153
767387992520,4cyclictest990566-21/usr/sbin/munin10:45:151
767387992520,3cyclictest762415-21kworker/u8:4+flush-259:007:25:161
767387992518,5cyclictest252-21systemd-journal07:19:301
767395992422,2cyclictest907663-21kworker/u8:2+flush-259:010:10:263
767395992420,3cyclictest850149-21kworker/u8:0+flush-259:008:30:283
767395992419,3cyclictest1016175-21kworker/u8:1+events_unbound11:25:223
767382992418,4cyclictest928293-21apt-get09:45:110
767382992417,5cyclictest907430-21latency_hist09:25:000
767395992320,2cyclictest828917-21kworker/u8:2+flush-259:008:45:153
767395992320,2cyclictest828917-21kworker/u8:2+events_unbound08:20:223
767395992319,2cyclictest813840-21kworker/u8:1+flush-259:008:10:163
767390992317,4cyclictest798338-21latency_hist07:40:012
767382992318,4cyclictest829276-21latency_hist08:10:010
767382992316,5cyclictest1067323-21ntpq11:55:210
767395992219,2cyclictest907663-21kworker/u8:2+flush-259:009:55:193
767395992218,3cyclictest907663-21kworker/u8:2+flush-259:010:00:313
767395992216,4cyclictest772744-21apt-get07:15:133
767390992218,3cyclictest850636-21apt-get08:30:172
767390992213,7cyclictest0-21swapper/211:35:002
767382992218,3cyclictest1010902-21kworker/u8:3+flush-259:012:25:270
767395992118,2cyclictest969918-21kworker/u8:4+events_unbound10:55:193
767395992118,2cyclictest828917-21kworker/u8:2+flush-259:008:25:223
767395992118,2cyclictest762415-21kworker/u8:4+events_unbound07:25:143
767395992117,3cyclictest969918-21kworker/u8:4+flush-259:010:30:283
767395992117,3cyclictest772500-21kworker/u8:0+events_unbound07:20:423
767395992117,3cyclictest1016175-21kworker/u8:1+flush-259:012:20:203
76739099219,10cyclictest500-21systemd-logind09:30:002
767390992115,4cyclictest777671-21latency_hist07:20:012
767387992116,4cyclictest828917-21kworker/u8:2+flush-259:008:35:021
767387992115,4cyclictest252-21systemd-journal08:51:321
767382992117,2cyclictest782849-21kworker/u8:2+flush-259:007:30:000
767395992018,2cyclictest1016175-21kworker/u8:1+flush-259:011:35:433
767395992017,2cyclictest1016175-21kworker/u8:1+flush-259:011:55:193
767395992016,3cyclictest1010876-21kworker/u8:2+events_unbound11:07:323
76739099209,10cyclictest1042701-21irqcore11:35:172
767390992015,4cyclictest907663-21kworker/u8:2+flush-259:010:45:002
767390992015,3cyclictest1063453-21diskstats11:55:142
767387992016,2cyclictest0-21swapper/111:00:001
767387992013,5cyclictest803544-21latency_hist07:45:021
767382992016,3cyclictest907663-21kworker/u8:2+flush-259:009:41:140
767382992016,3cyclictest876432-21apt-get08:55:100
767382992016,3cyclictest813840-21kworker/u8:1+events_unbound08:15:220
767382992014,5cyclictest1103889-21ntpq12:30:250
767395991917,2cyclictest1021574-21kworker/u8:0+events_unbound11:24:333
767395991917,2cyclictest1021574-21kworker/u8:0+events_unbound11:24:333
767395991916,2cyclictest907664-21kworker/u8:4+flush-259:009:40:233
767395991916,2cyclictest907663-21kworker/u8:2+events_unbound09:45:203
767395991915,3cyclictest954429-21kworker/u8:3+flush-259:010:20:193
767390991915,3cyclictest891568-21kworker/u8:2+flush-259:009:15:152
767387991915,3cyclictest907663-21kworker/u8:2+flush-259:009:45:321
767382991915,3cyclictest1010902-21kworker/u8:3+flush-259:012:20:170
767382991913,4cyclictest985187-21latency_hist10:40:000
767395991815,2cyclictest1016175-21kworker/u8:1+flush-259:011:30:253
767395991814,3cyclictest1010902-21kworker/u8:3+flush-259:012:15:173
767395991811,5cyclictest954191-21latency_hist10:10:013
76739099188,8cyclictest0-21swapper/209:35:012
76739099187,1cyclictest0-21swapper/208:00:192
76739099186,2cyclictest0-21swapper/212:30:232
767387991815,2cyclictest902267-21idleruntime-cro09:20:011
767387991813,3cyclictest828917-21kworker/u8:2+events_unbound08:25:191
767387991811,4cyclictest772481-21latency_hist07:15:021
767382991814,3cyclictest928055-21kworker/u8:0+flush-259:009:55:200
767382991814,3cyclictest907663-21kworker/u8:2+flush-259:010:15:170
767382991811,5cyclictest1072464-21ntpq12:00:230
767395991714,2cyclictest969918-21kworker/u8:4+events_unbound10:35:163
767395991714,2cyclictest894118-21kthreadcore09:10:183
76739099176,3cyclictest864914-21ntpq08:40:202
767390991714,2cyclictest907663-21kworker/u8:2+flush-259:010:25:162
767387991713,3cyclictest907663-21kworker/u8:2+flush-259:010:05:011
767387991713,3cyclictest840071-21apt-key08:20:011
767387991710,5cyclictest1006051-21/usr/sbin/munin11:00:161
767382991714,2cyclictest60350irq/132-enp0s29f1:rx-009:05:130
767382991713,3cyclictest1010902-21kworker/u8:3+events_unbound11:15:220
767382991713,2cyclictest828917-21kworker/u8:2+events_unbound08:15:000
767382991713,2cyclictest1-21systemd07:32:310
767382991711,4cyclictest954419-40mandb10:10:020
767395991612,3cyclictest1010902-21kworker/u8:3+flush-259:012:25:173
767395991610,4cyclictest911728-21ntpq09:25:203
767390991610,4cyclictest954419-40mandb10:10:022
767387991613,2cyclictest975544-21irqcore10:30:161
767387991612,3cyclictest1016175-21kworker/u8:1+flush-259:011:25:001
767387991612,3cyclictest1016175-21kworker/u8:1+flush-259:011:25:001
767387991611,3cyclictest252-21systemd-journal12:10:501
767387991611,3cyclictest1094027-21sleep12:25:001
767382991614,1cyclictest60350irq/132-enp0s29f1:rx-008:35:160
767382991614,1cyclictest60350irq/132-enp0s29f1:rx-007:35:280
767382991613,2cyclictest870965-21cron08:50:000
767382991613,2cyclictest60350irq/132-enp0s29f1:rx-010:40:210
767382991613,2cyclictest60350irq/132-enp0s29f1:rx-007:49:310
767382991612,3cyclictest813840-21kworker/u8:1+flush-259:008:25:270
767382991612,3cyclictest777964-21apt-get07:20:370
767382991612,3cyclictest1010902-21kworker/u8:3+flush-259:012:35:210
767382991611,3cyclictest840215-21gpgv08:20:010
767395991513,2cyclictest772500-21kworker/u8:0+flush-259:007:40:183
767395991513,2cyclictest1010902-21kworker/u8:3+events_unbound12:04:323
767395991513,2cyclictest1010902-21kworker/u8:3+events_unbound11:42:323
767395991512,2cyclictest1010902-21kworker/u8:3+events_unbound11:45:453
76739099157,1cyclictest1079783-21tr12:10:182
767390991512,2cyclictest990365-21kworker/u8:0+flush-259:010:50:242
767390991510,3cyclictest500-21systemd-logind12:20:012
76738799159,4cyclictest907664-21kworker/u8:4+flush-259:009:30:001
767387991512,2cyclictest1032368-21irqcore11:25:171
767387991511,3cyclictest969918-21kworker/u8:4+flush-259:010:25:361
767382991513,1cyclictest60350irq/132-enp0s29f1:rx-011:10:160
767382991513,1cyclictest60350irq/132-enp0s29f1:rx-011:08:330
767382991513,1cyclictest60350irq/132-enp0s29f1:rx-010:30:160
767382991513,1cyclictest60350irq/132-enp0s29f1:rx-007:55:120
767382991512,2cyclictest60350irq/132-enp0s29f1:rx-012:15:510
767382991512,2cyclictest60350irq/132-enp0s29f1:rx-009:10:260
767382991511,3cyclictest60350irq/132-enp0s29f1:rx-010:55:130
767382991511,2cyclictest0-21swapper/007:55:000
76739099149,3cyclictest1057468-21latency_hist11:50:012
76739099145,2cyclictest17550irq/130-nvme0q308:50:002
76739099144,2cyclictest1010157-21ntpq11:00:252
76738799149,3cyclictest954429-21kworker/u8:3+flush-259:010:25:001
76738799148,4cyclictest252-21systemd-journal08:59:591
76738799147,5cyclictest823302-21ntpq08:00:241
767387991412,1cyclictest0-21swapper/109:10:171
767387991410,3cyclictest907664-21kworker/u8:4+flush-259:009:40:251
767387991410,3cyclictest839654-21apt-get08:19:591
767387991410,3cyclictest1016175-21kworker/u8:1+flush-259:011:55:211
767382991413,1cyclictest60350irq/132-enp0s29f1:rx-009:51:230
767382991412,2cyclictest61150irq/133-enp0s29f1:tx-009:25:240
767382991412,2cyclictest60350irq/132-enp0s29f1:rx-010:29:310
767382991412,2cyclictest60350irq/132-enp0s29f1:rx-010:06:260
767382991412,1cyclictest60350irq/132-enp0s29f1:rx-010:50:260
767382991412,1cyclictest60350irq/132-enp0s29f1:rx-010:48:450
767382991412,1cyclictest60350irq/132-enp0s29f1:rx-010:00:250
767382991412,1cyclictest60350irq/132-enp0s29f1:rx-009:32:290
767382991412,1cyclictest60350irq/132-enp0s29f1:rx-008:30:300
767382991412,1cyclictest60350irq/132-enp0s29f1:rx-007:40:220
767382991412,1cyclictest60350irq/132-enp0s29f1:rx-007:10:230
767395991311,2cyclictest1016175-21kworker/u8:1+events_unbound12:05:483
767395991310,2cyclictest990557-21kworker/u8:3+flush-259:010:50:203
767395991310,2cyclictest828917-21kworker/u8:2+events_unbound08:18:313
76739099139,3cyclictest1010902-21kworker/u8:3+flush-259:012:35:252
76739099139,2cyclictest0-21swapper/208:20:002
76739099137,4cyclictest787992-21latency_hist07:30:022
767390991310,2cyclictest948099-21ntp_kernel_err10:00:232
767390991310,2cyclictest1020918-21unixbench_singl11:10:262
767390991310,1cyclictest0-21swapper/209:00:172
76738799139,3cyclictest1016175-21kworker/u8:1+flush-259:011:30:261
76738799138,3cyclictest1010902-21kworker/u8:3+flush-259:011:05:191
767387991310,2cyclictest963611-21ntp_kernel_err10:15:211
767387991310,2cyclictest949446-21fschecks_count10:05:141
76738299139,2cyclictest0-21swapper/010:20:140
767382991310,2cyclictest61150irq/133-enp0s29f1:tx-008:50:160
767382991310,2cyclictest60350irq/132-enp0s29f1:rx-011:35:160
767382991310,2cyclictest60350irq/132-enp0s29f1:rx-011:00:140
767382991310,2cyclictest60350irq/132-enp0s29f1:rx-008:00:140
76739099129,2cyclictest964918-21cpuspeed_turbos10:20:132
76739099129,2cyclictest841359-21sed08:20:192
76739099129,2cyclictest0-21swapper/211:50:212
76739099129,2cyclictest0-21swapper/211:45:002
76739099128,3cyclictest772500-21kworker/u8:0+flush-259:007:55:182
76739099127,4cyclictest772500-21kworker/u8:0+flush-259:007:20:412
767390991210,1cyclictest0-21swapper/210:50:002
76738799129,2cyclictest989847-21unixbench_multi10:40:261
76738799126,4cyclictest252-21systemd-journal12:20:001
76738799126,4cyclictest252-21systemd-journal07:59:591
76738299129,2cyclictest901413-21ntp_kernel_pll_09:15:220
76738299129,2cyclictest60350irq/132-enp0s29f1:rx-011:20:210
76738299129,2cyclictest60350irq/132-enp0s29f1:rx-011:20:200
76738299129,2cyclictest60350irq/132-enp0s29f1:rx-007:18:280
76738299124,5cyclictest252-21systemd-journal12:10:000
767382991211,1cyclictest60350irq/132-enp0s29f1:rx-011:52:550
767382991210,1cyclictest60350irq/132-enp0s29f1:rx-009:35:170
76739099118,2cyclictest927560-21unixbench_multi09:40:242
76739099117,3cyclictest793207-21awk07:35:002
76739099115,4cyclictest252-21systemd-journal08:30:002
76738799117,3cyclictest813840-21kworker/u8:1+flush-259:008:10:211
76738799117,3cyclictest782378-21unixbench_multi07:20:501
76738799117,3cyclictest762207-21kworker/u8:3+flush-259:007:50:241
76738799117,3cyclictest1010902-21kworker/u8:3+flush-259:012:35:201
76738799117,2cyclictest0-21swapper/112:30:171
76738799116,3cyclictest252-21systemd-journal09:38:311
76739099107,2cyclictest959713-21cpuspeed_turbos10:15:142
76739099107,2cyclictest809385-21irqcore07:50:162
76739099106,3cyclictest829542-21apt-get08:10:122
76739099105,4cyclictest1073352-21latency_hist12:05:012
76739099104,4cyclictest490-21dbus-daemon12:10:012
76738799108,1cyclictest0-21swapper/111:15:001
76738799107,2cyclictest881626-21cpuspeed_turbos09:00:151
76738799107,2cyclictest1010902-21kworker/u8:3+flush-259:011:50:121
76738299108,1cyclictest60350irq/132-enp0s29f1:rx-008:40:220
7673909997,1cyclictest0-21swapper/212:25:012
7673909997,1cyclictest0-21swapper/209:55:012
7673909996,2cyclictest0-21swapper/212:25:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional