You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-31 - 14:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sun Aug 31, 2025 00:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3930503994239,2cyclictest3969284-21kworker/u8:2+events_unbound22:55:221
3930503994135,4cyclictest4108988-21latency_hist22:55:011
3930503993936,2cyclictest3935491-21kworker/u8:1+events_unbound20:05:001
3930503993935,3cyclictest4010137-21kworker/u8:0+events_unbound21:55:151
3930503993633,2cyclictest3979584-21kworker/u8:0+flush-259:020:55:161
3930503993632,3cyclictest4134230-21kworker/u8:3+events_unbound00:35:271
3930503993632,3cyclictest4134230-21kworker/u8:3+events_unbound00:35:271
3930503993531,3cyclictest4134230-21kworker/u8:3+events_unbound00:20:161
3930503993330,2cyclictest3969284-21kworker/u8:2+events_unbound21:10:181
3930503993330,2cyclictest3969284-21kworker/u8:2+events_unbound21:00:131
3930503993129,1cyclictest4134230-21kworker/u8:3+events_unbound00:32:571
3930503993128,2cyclictest4124706-21kworker/u8:1+events_unbound23:50:241
3930503993128,2cyclictest4057311-21kworker/u8:4+events_unbound22:05:221
3930503993027,3cyclictest3969284-21kworker/u8:2+events_unbound21:15:141
3930501993024,5cyclictest3754036-21systemd-journal23:30:220
3930511992926,2cyclictest4124706-21kworker/u8:1+flush-259:023:40:003
3930507992923,4cyclictest3934228-21kworker/u8:0+flush-259:019:45:002
3930503992923,4cyclictest3000063-21rs:main1
3930503992826,2cyclictest3916749-21kworker/u8:2+flush-259:020:10:101
3930503992825,2cyclictest4134230-21kworker/u8:3+events_unbound23:40:231
3930503992824,3cyclictest4026017-21kworker/u8:3+events_unbound21:35:151
3930501992824,3cyclictest4134230-21kworker/u8:3+flush-259:000:25:000
3930511992721,4cyclictest3754036-21systemd-journal00:36:333
3930511992721,4cyclictest3754036-21systemd-journal00:36:323
3930507992722,4cyclictest4171076-21/usr/sbin/munin23:55:132
3930503992725,2cyclictest3969284-21kworker/u8:2+events_unbound20:50:141
3930501992724,2cyclictest4134230-21kworker/u8:3+flush-259:000:35:250
3930501992724,2cyclictest4134230-21kworker/u8:3+flush-259:000:35:240
3930507992619,5cyclictest4077742-21latency_hist22:25:002
3930503992623,2cyclictest3953869-21kworker/u8:1+events_unbound20:45:221
3930511992522,2cyclictest4010137-21kworker/u8:0+flush-259:021:50:183
3930511992521,3cyclictest4134230-21kworker/u8:3+flush-259:000:10:183
393051199251,22cyclictest1-21systemd23:20:003
3930503992523,2cyclictest4124706-21kworker/u8:1+events_unbound23:10:171
3930503992522,2cyclictest4010137-21kworker/u8:0+events_unbound21:25:251
3930503992520,2cyclictest4010137-21kworker/u8:0+events_unbound21:54:551
3930501992519,4cyclictest3935890-21latency_hist19:50:000
3930511992420,3cyclictest3969284-21kworker/u8:2+flush-259:022:00:153
3930507992420,3cyclictest3969284-21kworker/u8:2+flush-259:021:00:242
3930503992421,2cyclictest4124706-21kworker/u8:1+events_unbound23:35:211
3930503992421,2cyclictest4124706-21kworker/u8:1+events_unbound23:22:481
3930503992421,2cyclictest3969284-21kworker/u8:2+events_unbound22:40:161
3930503992420,3cyclictest4124706-21kworker/u8:1+events_unbound23:30:241
3930511992319,3cyclictest4057311-21kworker/u8:4+flush-259:022:10:163
3930507992318,3cyclictest3934228-21kworker/u8:0+flush-259:019:55:002
3930507992317,4cyclictest3969261-21latency_hist20:40:012
3930503992320,2cyclictest3938851-21kworker/u8:0+flush-259:020:20:181
3930503992317,4cyclictest4041371-21base6421:50:001
3930501992317,4cyclictest4180388-21ntpq00:00:410
3930511992218,3cyclictest4124706-21kworker/u8:1+flush-259:023:25:173
3930511992218,3cyclictest4057311-21kworker/u8:4+flush-259:022:55:263
3930511992217,4cyclictest4160571-21latency_hist23:45:023
3930507992219,2cyclictest4150248-21munin-plugin-st23:35:002
3930503992219,2cyclictest3953869-21kworker/u8:1+events_unbound20:44:521
3930503992219,2cyclictest3000063-21rs:main1
3930503992215,5cyclictest4103404-21latency_hist22:50:001
3930501992216,5cyclictest4134419-21apt-get23:16:200
3930511992118,3cyclictest4160588-21kworker/u8:2+flush-259:023:55:243
3930511992118,2cyclictest3969284-21kworker/u8:2+events_unbound23:05:143
3930511992117,3cyclictest4010137-21kworker/u8:0+flush-259:021:55:183
3930507992118,2cyclictest4119338-21kworker/u8:0+flush-259:023:10:272
3930507992117,3cyclictest3969284-21kworker/u8:2+flush-259:021:20:182
3930503992118,2cyclictest4176032-21kworker/u8:1+events_unbound00:05:281
3930503992118,2cyclictest3969284-21kworker/u8:2+events_unbound23:07:211
3930503992118,2cyclictest3969284-21kworker/u8:2+events_unbound21:20:241
3930503992118,2cyclictest3937997-21kworker/u8:4+events_unbound20:25:201
3930501992117,3cyclictest3969284-21kworker/u8:2+flush-259:022:40:200
3930511992017,2cyclictest3953869-21kworker/u8:1+flush-259:021:05:213
3930511992016,3cyclictest4134230-21kworker/u8:3+flush-259:023:20:213
3930511992016,3cyclictest4026017-21kworker/u8:3+events_unbound21:45:243
3930511992016,3cyclictest3927719-21kworker/u8:1+events_unbound19:10:003
3930503992017,2cyclictest4057311-21kworker/u8:4+events_unbound23:00:221
3930503992014,4cyclictest4035249-21ntpq21:40:231
3930503992014,4cyclictest4024861-21ntpq21:30:221
3930501992017,2cyclictest3969284-21kworker/u8:2+events_unbound22:50:150
3930501992016,3cyclictest4026017-21kworker/u8:3+events_unbound21:35:170
3930501992016,3cyclictest3969284-21kworker/u8:2+events_unbound22:07:390
3930501992016,3cyclictest3953869-21kworker/u8:1+flush-259:020:40:160
3930501992015,4cyclictest3937144-21turbostat.cron20:00:000
3930501992015,4cyclictest18754-21latency_hist00:35:010
3930501992014,4cyclictest4191961-21cron00:14:590
3930511991917,1cyclictest3979584-21kworker/u8:0+events_unbound20:55:243
3930511991916,2cyclictest3938851-21kworker/u8:0+events_unbound20:11:093
3930511991916,2cyclictest0-21swapper/322:10:013
3930511991914,4cyclictest3932961-21latency_hist19:30:003
3930511991914,3cyclictest3959007-21latency_hist20:30:013
3930507991915,3cyclictest4134230-21kworker/u8:3+flush-259:000:25:162
3930507991915,3cyclictest4041440-21latency_hist21:50:002
3930507991913,4cyclictest3933582-21latency_hist19:35:012
3930503991917,2cyclictest3938851-21kworker/u8:0+events_unbound20:15:131
3930503991917,1cyclictest3953869-21kworker/u8:1+events_unbound21:05:271
3930503991916,2cyclictest4160588-21kworker/u8:2+events_unbound23:55:241
3930503991916,2cyclictest4134230-21kworker/u8:3+events_unbound00:04:131
3930503991916,2cyclictest4010137-21kworker/u8:0+events_unbound22:04:211
3930503991916,2cyclictest3916749-21kworker/u8:2+events_unbound19:42:201
3930503991915,2cyclictest3935491-21kworker/u8:1+events_unbound19:47:201
3930503991913,4cyclictest4083139-21/usr/sbin/munin22:30:141
3930503991912,5cyclictest4067416-21latency_hist22:15:001
3930501991915,3cyclictest3969284-21kworker/u8:2+events_unbound23:00:120
3930501991914,4cyclictest3979801-21apt-get20:50:120
3930511991815,1cyclictest0-21swapper/320:05:003
3930507991814,3cyclictest4057311-21kworker/u8:4+flush-259:022:05:252
3930507991814,3cyclictest4010137-21kworker/u8:0+flush-259:021:55:172
3930503991815,2cyclictest8417-21kworker/u8:2+events_unbound00:25:241
3930503991815,2cyclictest4057311-21kworker/u8:4+events_unbound22:19:581
3930501991815,2cyclictest3969284-21kworker/u8:2+flush-259:022:45:260
3930501991815,2cyclictest3937997-21kworker/u8:4+events_unbound20:26:480
3930501991814,3cyclictest4134230-21kworker/u8:3+flush-259:023:40:240
3930501991814,3cyclictest3938851-21kworker/u8:0+events_unbound20:35:190
3930511991713,3cyclictest3969284-21kworker/u8:2+flush-259:021:20:213
3930511991713,3cyclictest3953869-21kworker/u8:1+flush-259:021:00:143
3930511991712,4cyclictest4057311-21kworker/u8:4+flush-259:022:30:153
3930511991710,5cyclictest4185545-21ntpq00:05:243
3930507991714,2cyclictest4176032-21kworker/u8:1+flush-259:000:04:592
3930507991714,2cyclictest3953869-21kworker/u8:1+flush-259:020:30:212
3930507991712,3cyclictest4078002-21apt-get22:25:102
3930507991711,4cyclictest4124480-21latency_hist23:10:012
393050799170,15cyclictest1-21systemd22:44:212
3930503991714,2cyclictest4057311-21kworker/u8:4+events_unbound22:20:171
3930503991714,2cyclictest3953869-21kworker/u8:1+events_unbound20:39:511
3930503991714,2cyclictest3937997-21kworker/u8:4+events_unbound20:06:411
3930503991714,2cyclictest3936519-21kworker/u8:3+events_unbound19:59:561
3930503991713,3cyclictest3953869-21kworker/u8:1+events_unbound20:30:131
3930501991714,2cyclictest3969284-21kworker/u8:2+flush-259:021:09:590
3930501991713,3cyclictest3969284-21kworker/u8:2+events_unbound20:45:140
3930501991713,3cyclictest3953869-21kworker/u8:1+events_unbound20:30:510
3930511991612,3cyclictest3969284-21kworker/u8:2+events_unbound22:25:173
3930511991612,3cyclictest3938851-21kworker/u8:0+flush-259:020:20:163
3930511991611,3cyclictest3969685-21diskstats20:40:143
3930511991610,4cyclictest4169931-21ntpq23:50:203
3930507991613,2cyclictest4041212-21apt-get21:50:032
3930507991613,2cyclictest4026017-21kworker/u8:3+events_unbound21:45:002
3930507991612,3cyclictest4134230-21kworker/u8:3+flush-259:000:05:262
3930507991612,3cyclictest4057311-21kworker/u8:4+flush-259:022:10:242
3930507991611,3cyclictest3935491-21kworker/u8:1+flush-259:019:50:002
3930503991615,1cyclictest4124706-21kworker/u8:1+events_unbound23:25:181
3930503991614,2cyclictest3929396-21kworker/u8:4+events_unbound19:32:191
3930503991612,3cyclictest4124706-21kworker/u8:1+flush-259:023:49:271
3930503991612,3cyclictest3916749-21kworker/u8:2+events_unbound19:38:441
3930501991614,2cyclictest61450irq/134-enp0s29f1:tx-022:35:180
3930501991613,2cyclictest4160588-21kworker/u8:2+events_unbound23:57:230
3930501991613,2cyclictest4134230-21kworker/u8:3+events_unbound00:05:250
3930501991612,3cyclictest4170658-21kworker/u8:0+events_unbound00:15:140
3930501991612,3cyclictest4010137-21kworker/u8:0+events_unbound21:50:160
3930501991612,3cyclictest3969284-21kworker/u8:2+flush-259:021:30:260
3930501991612,3cyclictest3969284-21kworker/u8:2+flush-259:021:15:110
3930501991612,3cyclictest3937997-21kworker/u8:4+events_unbound20:12:440
3930501991612,3cyclictest3936519-21kworker/u8:3+flush-259:020:10:000
3930501991611,4cyclictest4140238-21apt-get23:25:110
3930501991611,4cyclictest3969284-21kworker/u8:2+events_unbound23:05:120
3930501991610,4cyclictest4071672-21ntpq22:15:210
393051199158,5cyclictest3969284-21kworker/u8:2+flush-259:021:35:003
3930511991513,1cyclictest4026017-21kworker/u8:3+events_unbound21:35:213
3930511991512,2cyclictest4134230-21kworker/u8:3+flush-259:000:25:133
3930511991512,2cyclictest4134230-21kworker/u8:3+flush-259:000:15:153
3930511991511,3cyclictest4134230-21kworker/u8:3+flush-259:000:20:213
393050799159,4cyclictest3754036-21systemd-journal22:57:442
3930507991512,2cyclictest0-21swapper/219:14:592
3930507991510,3cyclictest3984661-21cron20:55:002
3930503991512,2cyclictest3929396-21kworker/u8:4+flush-259:019:17:061
3930503991512,2cyclictest3929396-21kworker/u8:4+events_unbound19:23:081
3930503991510,3cyclictest3969284-21kworker/u8:2+flush-259:022:25:231
393050199159,4cyclictest4004429-21ntpq21:10:250
3930501991514,1cyclictest60650irq/133-enp0s29f1:rx-022:00:330
3930501991513,2cyclictest60650irq/133-enp0s29f1:rx-021:40:360
3930501991513,1cyclictest60650irq/133-enp0s29f1:rx-023:35:150
3930501991513,1cyclictest60650irq/133-enp0s29f1:rx-022:25:230
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-022:56:080
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-021:50:010
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-020:55:210
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-020:04:460
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-019:41:590
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-019:29:570
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-019:25:000
3930501991512,2cyclictest60650irq/133-enp0s29f1:rx-019:12:100
3930501991512,2cyclictest4134230-21kworker/u8:3+events_unbound23:45:200
3930501991512,2cyclictest4124706-21kworker/u8:1+events_unbound23:20:200
3930501991512,2cyclictest4124706-21kworker/u8:1+events_unbound23:10:140
3930501991511,3cyclictest4010137-21kworker/u8:0+events_unbound21:55:210
3930501991511,3cyclictest3938851-21kworker/u8:0+events_unbound20:19:500
3930511991412,1cyclictest4134230-21kworker/u8:3+events_unbound23:45:133
3930511991412,1cyclictest0-21swapper/319:54:593
3930511991411,2cyclictest4124706-21kworker/u8:1+events_unbound23:10:183
3930511991411,2cyclictest4072569-21idleruntime-cro22:20:003
3930511991411,1cyclictest0-21swapper/300:04:593
3930511991410,3cyclictest3936519-21kworker/u8:3+events_unbound20:15:283
393050799146,3cyclictest4145125-21idleruntime-cro23:30:012
3930507991410,3cyclictest3938851-21kworker/u8:0+flush-259:020:15:252
3930507991410,2cyclictest4088325-21apt-get22:35:122
393050799140,6cyclictest3934219-21munin-run19:39:592
393050399145,1cyclictest191rcu_preempt00:10:011
3930503991411,2cyclictest3934228-21kworker/u8:0+events_unbound19:53:331
3930503991411,2cyclictest3916749-21kworker/u8:2+events_unbound19:10:091
3930503991410,3cyclictest3931933-21kworker/u8:3+events_unbound19:29:401
3930501991412,2cyclictest60650irq/133-enp0s29f1:rx-022:20:020
3930501991412,2cyclictest60650irq/133-enp0s29f1:rx-022:10:140
3930501991412,1cyclictest60650irq/133-enp0s29f1:rx-021:02:540
3930501991411,2cyclictest4124706-21kworker/u8:1+events_unbound23:50:130
3930501991411,2cyclictest4083586-21irqcore22:30:160
3930501991411,2cyclictest3933568-21cron19:34:590
393051199139,3cyclictest3984665-21sh20:55:003
393051199138,4cyclictest4134230-21kworker/u8:3+flush-259:023:30:243
3930511991311,1cyclictest3933584-21latency_hist19:35:003
3930511991311,1cyclictest0-21swapper/322:50:003
3930511991310,2cyclictest4093506-21cpuspeed_turbos22:40:133
3930511991310,2cyclictest3969284-21kworker/u8:2+events_unbound23:00:193
3930511991310,2cyclictest16122-21kthreadcore00:30:153
393050799139,3cyclictest3938851-21kworker/u8:0+flush-259:020:20:152
3930507991310,2cyclictest4155014-21unixbench_singl23:35:232
3930501991311,1cyclictest60650irq/133-enp0s29f1:rx-019:17:330
3930501991311,1cyclictest0-21swapper/019:55:000
3930501991310,2cyclictest60650irq/133-enp0s29f1:rx-021:25:160
3930501991310,2cyclictest60650irq/133-enp0s29f1:rx-019:38:270
393051199129,2cyclictest4103484-21kworker/u8:1+flush-259:022:50:163
393051199129,2cyclictest3964557-21fschecks_count20:35:143
393050799129,2cyclictest4187157-21cpuspeed_turbos00:10:122
393050799129,2cyclictest4130288-21irqcore23:15:172
393050799129,2cyclictest4067852-21fschecks_count22:15:142
393050799129,2cyclictest17749-21cut00:30:172
393050799129,2cyclictest0-21swapper/220:10:002
393050799129,1cyclictest0-21swapper/221:15:002
393050799128,3cyclictest4134230-21kworker/u8:3+flush-259:000:20:252
393050799128,3cyclictest4124706-21kworker/u8:1+flush-259:023:50:172
393050799126,5cyclictest4088066-21latency_hist22:35:012
393050799121,3cyclictest3931716-21munin-plugin-st19:20:002
393050199129,2cyclictest60650irq/133-enp0s29f1:rx-020:21:300
3930501991210,1cyclictest60650irq/133-enp0s29f1:rx-000:25:250
393051199118,2cyclictest3969284-21kworker/u8:2+flush-259:021:15:183
393051199118,2cyclictest3953869-21kworker/u8:1+flush-259:020:30:213
393051199117,3cyclictest3969284-21kworker/u8:2+events_unbound21:10:283
393050799119,1cyclictest837-21snmpd19:24:442
393050799110,9cyclictest4000145-21awk21:10:012
393050199119,1cyclictest60650irq/133-enp0s29f1:rx-021:20:150
393051199108,1cyclictest0-21swapper/319:45:013
393051199106,3cyclictest3916749-21kworker/u8:2+flush-259:019:12:183
393051199106,2cyclictest0-21swapper/319:25:003
393050799104,4cyclictest3984941-21apt-get20:55:122
393050799103,6cyclictest3936519-21kworker/u8:3+flush-259:019:55:312
393050799101,7cyclictest3932947-21awk19:30:002
393050399103,2cyclictest32-21ksoftirqd/119:10:001
39305119996,2cyclictest0-21swapper/320:10:003
39305119995,2cyclictest3754036-21systemd-journal19:59:553
39305079996,2cyclictest4052248-21cpuspeed_turbos22:00:112
39305079994,4cyclictest2181-21memory00:15:202
39305119975,1cyclictest0-21swapper/319:17:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional