You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-12 - 00:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Fri Sep 11, 2026 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
356657721330,4sleep13056-21PK-Backend08:05:011
357899521230,4sleep03056-21PK-Backend08:40:000
357129921180,5sleep13056-21PK-Backend08:15:201
357258821170,4sleep0131ksoftirqd/008:20:160
360040121140,4sleep30-21swapper/309:40:023
354751421080,3sleep10-21swapper/107:10:151
365238521010,4sleep13056-21PK-Backend12:05:131
35459612820,18sleep10-21swapper/107:05:211
354662828039,13sleep20-21swapper/207:08:042
139099760,61rtkit-daemon0-21swapper/007:09:070
354607527045,16sleep30-21swapper/307:05:243
35953792600,5sleep30-21swapper/309:25:153
36504892510,2sleep20-21swapper/212:00:122
35792142500,4sleep126-21ksoftirqd/108:40:141
36039652490,2sleep33603635-21apt-get09:50:013
139099430,6rtkit-daemon3056-21PK-Backend11:43:171
139099430,6rtkit-daemon3056-21PK-Backend09:05:371
36488922420,2sleep30-21swapper/311:55:123
35723642420,1sleep13572368-21tr08:20:131
139099410,5rtkit-daemon3056-21PK-Backend10:57:351
139099390,4rtkit-daemon3056-21PK-Backend09:32:201
139099390,4rtkit-daemon3056-21PK-Backend08:09:531
139099380,4rtkit-daemon3056-21PK-Backend07:23:091
139099380,4rtkit-daemon3056-21PK-Backend07:23:091
139099380,3rtkit-daemon3056-21PK-Backend11:34:571
139099370,4rtkit-daemon3056-21PK-Backend09:57:321
139099370,3rtkit-daemon3056-21PK-Backend10:04:301
139099370,3rtkit-daemon0-21swapper/109:15:281
139099360,3rtkit-daemon3056-21PK-Backend11:02:521
139099360,3rtkit-daemon3056-21PK-Backend10:18:381
139099360,3rtkit-daemon0-21swapper/109:14:531
354691799336,9cyclictest3648335-21apt-config11:55:002
354691999320,6cyclictest3056-21PK-Backend10:25:003
3546919993016,12cyclictest3559562-21unin-run07:45:003
354691799306,9cyclictest3559536-21munin-run07:45:002
354691799273,15cyclictest3653548-21cat12:10:002
354691799272,15cyclictest0-21swapper/210:10:162
354691799272,15cyclictest0-21swapper/209:05:092
354691999269,5cyclictest42-21ksoftirqd/311:55:013
3546919992614,11cyclictest42-21ksoftirqd/312:40:013
354691999260,7cyclictest42-21ksoftirqd/310:50:013
354691799269,15cyclictest3604333-21cpuspeed_turbos09:50:162
354691799267,17cyclictest3575865-21cstates08:30:132
354691799264,9cyclictest3616780-21cstates10:25:122
354691799263,7cyclictest3568723-21apt08:10:112
3546917992618,6cyclictest3592962-21cron09:20:002
354691799260,10cyclictest35834422sensors08:50:192
354691999250,23cyclictest3611468-21df_inode10:10:173
354691799254,8cyclictest3634633-21irqstats11:15:152
354691799253,8cyclictest3587544-21systemctl09:05:012
354691799252,9cyclictest0-21swapper/212:15:032
354691799252,14cyclictest0-21swapper/210:20:032
354691799252,12cyclictest0-21swapper/211:25:122
354691799251,7cyclictest0-21swapper/211:20:002
3546917992514,7cyclictest3599932-21sleep09:40:002
354691999248,14cyclictest3609089-21grep10:05:013
3546919992411,3cyclictest42-21ksoftirqd/308:10:033
354691999240,6cyclictest3577965-21/usr/sbin/munin08:35:173
354691999240,6cyclictest3056-21PK-Backend10:28:223
354691799244,8cyclictest3619026-21ntp_states10:30:192
354691799244,6cyclictest2156-21gnome-shell10:50:092
354691799243,6cyclictest3549714-21diskstats07:15:132
354691799243,6cyclictest2156-21gnome-shell09:26:092
354691799243,12cyclictest0-21swapper/208:35:162
354691799242,13cyclictest0-21swapper/209:55:122
3546917992415,5cyclictest3551079-21apt07:20:102
3546917992415,5cyclictest3551079-21apt07:20:102
3546917992414,7cyclictest3603668-21/usr/sbin/munin09:50:012
354691799241,14cyclictest0-21swapper/209:40:142
35850642230,3sleep20-21swapper/208:55:182
35683812230,3sleep00-21swapper/008:10:000
35478212230,3sleep0161rcuc/007:10:180
354691999230,7cyclictest42-21ksoftirqd/307:21:033
354691999230,7cyclictest42-21ksoftirqd/307:21:033
354691999230,21cyclictest3635136-21wc11:15:213
354691799234,6cyclictest3643459-21diskstats11:40:132
354691799233,5cyclictest3557812-21munin-run07:40:002
354691799233,12cyclictest0-21swapper/209:30:182
354691799232,12cyclictest0-21swapper/211:00:172
3546917992316,3cyclictest3641085-21sh11:35:002
3546917992315,4cyclictest3564128-21python307:55:222
3546917992314,7cyclictest3581701-21chrt08:45:202
3546917992313,7cyclictest3590971-21apt-get09:15:002
354691799230,21cyclictest3623935-21cat10:45:142
3546910992320,2cyclictest0-21swapper/011:07:300
3546910992314,7cyclictest3616914-21df10:25:120
354691999228,5cyclictest42-21ksoftirqd/312:10:003
354691999226,8cyclictest42-21ksoftirqd/310:54:353
354691999223,5cyclictest42-21ksoftirqd/309:05:123
354691999223,10cyclictest42-21ksoftirqd/311:20:013
3546919992216,5cyclictest42-21ksoftirqd/308:45:003
3546919992213,7cyclictest3599911-21apt-get09:40:003
354691999221,20cyclictest3629698-21ntp_states11:00:203
354691999220,10cyclictest42-21ksoftirqd/309:15:163
354691799228,12cyclictest3614325-21unin-run10:20:002
354691799223,7cyclictest0-21swapper/207:50:222
3546917992212,7cyclictest3568228-21unin-run08:10:012
3546917992211,8cyclictest36226202chrt10:40:212
354691799220,20cyclictest0-21swapper/207:45:422
354691499220,20cyclictest3056-21PK-Backend09:20:401
3546910992220,1cyclictest0-21swapper/012:11:380
3546910992220,1cyclictest0-21swapper/008:16:360
3546910992220,1cyclictest0-21swapper/008:01:560
3546910992219,2cyclictest3056-21PK-Backend11:50:100
3546910992219,2cyclictest3056-21PK-Backend07:52:160
36243482210,5sleep00-21swapper/010:45:210
354691999219,11cyclictest3554263-21cron07:30:003
354691999216,8cyclictest42-21ksoftirqd/312:03:563
354691999215,8cyclictest42-21ksoftirqd/312:29:233
354691999215,8cyclictest42-21ksoftirqd/310:36:033
354691999215,8cyclictest42-21ksoftirqd/309:12:453
354691999215,8cyclictest42-21ksoftirqd/308:50:563
354691999215,8cyclictest42-21ksoftirqd/308:06:053
354691999213,5cyclictest3555596-21sed07:30:223
3546919992117,3cyclictest42-21ksoftirqd/310:20:003
3546919992114,5cyclictest3646275-21cron11:50:003
354691999210,6cyclictest3056-21PK-Backend10:31:543
3546917992119,1cyclictest0-21swapper/211:48:562
3546917992119,1cyclictest0-21swapper/210:56:512
3546917992119,1cyclictest0-21swapper/210:07:232
354691799211,8cyclictest0-21swapper/210:35:172
3546917992115,3cyclictest3632149-21perf11:10:002
3546917992114,5cyclictest3662001-21wc12:30:282
3546917992114,4cyclictest3642808-21cron11:40:002
3546917992113,5cyclictest3593916-21needreboot09:20:172
3546917992113,5cyclictest3556013-21munin-run07:35:002
3546917992112,7cyclictest3660699-21idleruntime-cro12:29:592
3546917992111,7cyclictest3658874-21sshd12:24:242
3546917992110,9cyclictest3571033-21chrt08:15:172
3546917992110,7cyclictest3648014-21apt-get11:55:032
3546914992115,5cyclictest3624982-21systemd-run10:50:011
3546914992112,7cyclictest3554285-21cron07:30:001
3546914992110,7cyclictest3618033-21sh10:29:591
354691499210,6cyclictest3056-21PK-Backend08:50:011
354691499210,19cyclictest3056-21PK-Backend12:25:141
354691499210,19cyclictest3056-21PK-Backend12:21:491
354691499210,19cyclictest3056-21PK-Backend12:17:371
354691499210,11cyclictest3662421-21sh12:34:591
3546910992119,1cyclictest3056-21PK-Backend07:36:140
3546910992119,1cyclictest0-21swapper/012:31:060
3546910992119,1cyclictest0-21swapper/012:24:160
3546910992119,1cyclictest0-21swapper/012:08:140
3546910992119,1cyclictest0-21swapper/011:34:480
3546910992119,1cyclictest0-21swapper/011:28:330
3546910992119,1cyclictest0-21swapper/011:24:550
3546910992119,1cyclictest0-21swapper/010:15:530
3546910992119,1cyclictest0-21swapper/009:59:440
3546910992119,1cyclictest0-21swapper/008:41:580
3546910992119,1cyclictest0-21swapper/008:12:370
3546910992117,3cyclictest3628698-21munin-run11:00:010
3546910992116,4cyclictest3618644-21expr10:30:130
354691099210,20cyclictest0-21swapper/012:15:520
354691099210,20cyclictest0-21swapper/011:36:380
354691099210,20cyclictest0-21swapper/011:15:490
354691099210,20cyclictest0-21swapper/010:43:590
139099210,14rtkit-daemon0-21swapper/007:27:420
354691999205,7cyclictest42-21ksoftirqd/307:10:073
354691999204,8cyclictest42-21ksoftirqd/308:30:033
354691999204,8cyclictest42-21ksoftirqd/307:35:463
354691999203,8cyclictest42-21ksoftirqd/308:00:323
354691999203,2cyclictest141rcu_preempt08:17:203
354691999201,9cyclictest0-21swapper/311:05:353
3546919992018,1cyclictest0-21swapper/310:59:193
3546919992016,3cyclictest411rcuc/307:16:093
3546919992015,4cyclictest42-21ksoftirqd/310:10:003
3546919992014,4cyclictest3593943-21/usr/sbin/munin09:20:183
3546919992012,6cyclictest3574401-21/usr/sbin/munin08:25:173
354691999200,6cyclictest3621758-21apt10:40:113
354691999200,1cyclictest0-21swapper/312:13:273
354691999200,1cyclictest0-21swapper/311:27:173
354691999200,1cyclictest0-21swapper/309:33:273
354691999200,19cyclictest0-21swapper/312:33:553
354691999200,19cyclictest0-21swapper/312:16:033
354691999200,19cyclictest0-21swapper/311:40:003
354691999200,19cyclictest0-21swapper/311:32:453
354691999200,19cyclictest0-21swapper/309:50:443
354691999200,19cyclictest0-21swapper/307:56:213
354691799209,8cyclictest3572808-21missed_timers08:20:182
3546917992013,3cyclictest3662686-21apt12:35:112
3546917992012,6cyclictest3574933-21wc08:25:222
354691799201,1cyclictest0-21swapper/211:13:552
3546917992011,6cyclictest3579099-21/usr/sbin/munin08:40:122
354691799200,19cyclictest0-21swapper/210:00:072
3546914992017,2cyclictest3056-21PK-Backend10:05:491
3546914992011,7cyclictest26-21ksoftirqd/108:15:001
354691499200,1cyclictest3056-21PK-Backend09:44:061
354691499200,1cyclictest3056-21PK-Backend08:28:001
354691499200,1cyclictest3056-21PK-Backend07:36:301
354691499200,19cyclictest3056-21PK-Backend11:21:401
354691499200,19cyclictest3056-21PK-Backend11:16:121
354691499200,19cyclictest3056-21PK-Backend08:50:121
354691499200,19cyclictest0-21swapper/111:25:181
354691499200,19cyclictest0-21swapper/109:50:551
354691499200,18cyclictest3056-21PK-Backend09:39:531
354691499200,18cyclictest3056-21PK-Backend09:25:391
354691499200,11cyclictest26-21ksoftirqd/110:55:011
3546910992018,1cyclictest3056-21PK-Backend08:45:220
3546910992018,1cyclictest3056-21PK-Backend07:45:270
3546910992018,1cyclictest0-21swapper/011:44:410
3546910992018,1cyclictest0-21swapper/010:05:110
3546910992018,1cyclictest0-21swapper/009:46:530
3546910992018,1cyclictest0-21swapper/009:43:290
3546910992018,1cyclictest0-21swapper/009:35:510
3546910992018,1cyclictest0-21swapper/009:29:020
3546910992018,1cyclictest0-21swapper/008:56:170
3546910992018,1cyclictest0-21swapper/008:54:420
354691099200,19cyclictest3056-21PK-Backend12:04:020
354691099200,19cyclictest0-21swapper/010:01:340
354691099200,19cyclictest0-21swapper/007:24:040
354691099200,19cyclictest0-21swapper/007:24:040
139099200,14rtkit-daemon0-21swapper/009:06:300
36628782190,3sleep13056-21PK-Backend12:35:111
36492212190,4chrt3056-21PK-Backend11:55:170
354691999194,7cyclictest42-21ksoftirqd/307:50:133
354691999191,6cyclictest3584318-21apt08:55:103
3546919991915,3cyclictest411rcuc/312:23:253
3546919991915,3cyclictest411rcuc/309:03:523
3546919991911,6cyclictest3633469-21sed11:10:203
354691999190,1cyclictest0-21swapper/311:36:233
354691799199,8cyclictest3553399-21wc07:25:172
3546917991912,5cyclictest3565130-21grep08:00:112
3546917991911,6cyclictest3653726-21/usr/sbin/munin12:10:232
354691499199,4cyclictest26-21ksoftirqd/110:30:091
3546914991917,1cyclictest3056-21PK-Backend12:14:121
3546914991917,1cyclictest3056-21PK-Backend10:21:441
3546914991917,1cyclictest3056-21PK-Backend10:12:521
3546914991917,1cyclictest3056-21PK-Backend07:50:571
3546914991912,2cyclictest26-21ksoftirqd/111:59:391
3546914991911,6cyclictest3621489-21cron10:40:001
354691499190,1cyclictest3056-21PK-Backend11:38:361
354691499190,1cyclictest3056-21PK-Backend08:56:201
354691499190,1cyclictest3056-21PK-Backend07:18:121
354691499190,1cyclictest0-21swapper/107:40:431
354691499190,18cyclictest3056-21PK-Backend12:04:581
354691499190,18cyclictest3056-21PK-Backend09:47:301
354691499190,18cyclictest0-21swapper/111:50:191
3546910991917,1cyclictest3056-21PK-Backend09:15:370
3546910991917,1cyclictest0-21swapper/012:26:530
3546910991917,1cyclictest0-21swapper/010:37:290
3546910991917,1cyclictest0-21swapper/010:14:040
3546910991917,1cyclictest0-21swapper/009:54:310
3546910991917,1cyclictest0-21swapper/007:57:300
3546910991917,1cyclictest0-21swapper/007:16:060
354691099190,1cyclictest0-21swapper/011:46:310
354691099190,1cyclictest0-21swapper/011:14:480
354691099190,1cyclictest0-21swapper/011:01:500
354691099190,1cyclictest0-21swapper/010:22:560
354691099190,1cyclictest0-21swapper/009:04:010
354691099190,18cyclictest0-21swapper/008:30:220
354691999182,14cyclictest0-21swapper/309:55:123
3546919991815,2cyclictest411rcuc/308:49:323
3546919991815,2cyclictest411rcuc/308:20:373
3546914991816,1cyclictest3056-21PK-Backend09:04:041
3546914991815,2cyclictest3056-21PK-Backend08:39:371
3546914991810,6cyclictest26-21ksoftirqd/111:10:071
3546910991816,1cyclictest3056-21PK-Backend07:42:020
3546910991816,1cyclictest0-21swapper/012:36:060
3546910991816,1cyclictest0-21swapper/009:32:270
3546919991714,2cyclictest411rcuc/307:49:133
3546917991711,4cyclictest3547330-21/usr/sbin/munin07:10:132
354691499179,2cyclictest26-21ksoftirqd/108:30:111
3546914991710,5cyclictest35546382sleep107:30:121
354691499170,6cyclictest3056-21PK-Backend07:55:231
3546910991715,1cyclictest3056-21PK-Backend10:52:300
3546910991715,1cyclictest0-21swapper/009:24:500
3546910991713,2cyclictest3556003-21sh07:35:000
354691499168,2cyclictest26-21ksoftirqd/111:45:151
3546914991614,1cyclictest3056-21PK-Backend10:41:451
3546914991511,2cyclictest26-21ksoftirqd/107:45:561
3546914991510,3cyclictest3056-21PK-Backend11:06:581
354691099159,5cyclictest3574058-21tr08:25:120
3546910991513,1cyclictest3056-21PK-Backend09:14:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional