You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 15:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Wed Apr 29, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139752238163,23sleep30-21swapper/307:05:053
158032231161,56sleep20-21swapper/207:08:132
137652229164,52sleep10-21swapper/107:05:021
158612202165,25sleep00-21swapper/007:08:590
1521021780,5sleep21603199cyclictest12:00:122
3264421600,3sleep21603199cyclictest07:45:122
2830521550,4sleep11602599cyclictest08:50:001
16031995147,3cyclictest0-21swapper/207:58:492
156242460,1sleep30-21swapper/309:33:573
290932420,1sleep10-21swapper/110:05:011
16017993711,13cyclictest23902-21munin-run08:40:010
1603199341,21cyclictest0-21swapper/207:27:032
1603199341,14cyclictest10366-21munin-run11:50:012
1603199331,19cyclictest0-21swapper/209:50:172
1603199321,20cyclictest0-21swapper/208:53:412
1601799325,13cyclictest25150-21munin-run11:10:010
1603899311,9cyclictest16460-21unixbench-2d12:00:243
1603199311,20cyclictest0-21swapper/211:23:142
1603199311,20cyclictest0-21swapper/207:50:132
1603199311,16cyclictest0-21swapper/211:39:532
1603199311,15cyclictest0-21swapper/211:41:052
1603899302,14cyclictest0-21swapper/311:14:573
1603199291,11cyclictest0-21swapper/210:25:092
1603199290,13cyclictest0-21swapper/210:04:272
1602599291,18cyclictest0-21swapper/112:16:411
1603899281,13cyclictest3339-21sort10:20:013
1603899281,10cyclictest12020-21grep09:25:153
1603199281,13cyclictest0-21swapper/208:21:372
1603199281,12cyclictest0-21swapper/210:52:392
1603199281,12cyclictest0-21swapper/210:30:252
1603199281,12cyclictest0-21swapper/208:27:512
1603199281,11cyclictest0-21swapper/208:32:312
1603199280,2cyclictest0-21swapper/207:42:332
1603199280,11cyclictest0-21swapper/209:55:372
1602599281,8cyclictest0-21swapper/110:08:111
1601799282,12cyclictest24322-21kworker/0:110:03:110
181162270,11sleep07-21ksoftirqd/008:25:150
1603899271,19cyclictest0-21swapper/310:31:113
1603899271,17cyclictest17256-21switchtime09:35:243
1603899271,12cyclictest26709-21latency_hist10:00:013
1603899270,17cyclictest0-21swapper/312:31:573
1603899270,16cyclictest0-21swapper/309:23:413
1603199271,13cyclictest0-21swapper/210:22:092
1603199271,13cyclictest0-21swapper/210:22:092
1603199271,12cyclictest0-21swapper/211:26:332
1603199271,11cyclictest0-21swapper/211:12:412
1603199271,10cyclictest0-21swapper/211:16:372
1602599273,16cyclictest0-21swapper/110:37:011
1601799277,17cyclictest20743-21munin-run11:00:010
1601799271,14cyclictest22323-21munin-run09:50:010
1603899268,7cyclictest9887-21unixbench_multi11:45:253
1603899266,10cyclictest87350irq/126-enp1s0-07:15:163
1603899262,12cyclictest0-21swapper/309:12:143
1603899262,10cyclictest0-21swapper/311:57:003
1603899260,19cyclictest26741-21kworker/u8:011:33:483
1603899260,19cyclictest26741-21kworker/u8:011:33:473
1603899260,17cyclictest15026-21expr10:45:153
1603899260,10cyclictest6847-21gltestperf11:40:153
1603199261,13cyclictest0-21swapper/212:25:132
1603199261,13cyclictest0-21swapper/211:04:372
1603199261,13cyclictest0-21swapper/210:38:052
1603199261,13cyclictest0-21swapper/208:56:132
1603199261,12cyclictest0-21swapper/212:15:452
1603199261,12cyclictest0-21swapper/211:05:532
1603199261,11cyclictest0-21swapper/212:35:152
1603199261,11cyclictest0-21swapper/212:31:372
1603199261,11cyclictest0-21swapper/210:10:172
1603199261,10cyclictest0-21swapper/208:18:592
1603199260,9cyclictest0-21swapper/208:46:272
1603199260,11cyclictest0-21swapper/207:22:472
1603899253,15cyclictest0-21swapper/311:35:193
1603899252,6cyclictest0-21swapper/312:07:303
1603899252,12cyclictest0-21swapper/311:50:523
1603899251,9cyclictest0-21swapper/310:06:363
1603899251,14cyclictest24609-21cat09:55:023
1603899251,10cyclictest0-21swapper/312:20:233
1603899251,10cyclictest0-21swapper/308:57:153
1603199251,12cyclictest0-21swapper/209:02:332
1603199251,11cyclictest0-21swapper/209:39:212
1603199251,10cyclictest0-21swapper/209:33:572
1603199251,10cyclictest0-21swapper/209:26:132
1603199251,10cyclictest0-21swapper/208:36:032
1602599251,17cyclictest0-21swapper/109:14:171
1603899246,7cyclictest0-21swapper/310:25:153
1603899244,3cyclictest0-21swapper/310:59:113
1603899242,6cyclictest0-21swapper/310:44:483
1603899242,6cyclictest0-21swapper/310:36:253
1603899242,11cyclictest87350irq/126-enp1s0-09:40:193
1603899241,11cyclictest4731-21diskmemload10:51:023
1603899241,11cyclictest0-21swapper/310:22:353
1603899241,11cyclictest0-21swapper/310:22:353
1603899240,16cyclictest87350irq/126-enp1s0-12:13:493
1603899240,16cyclictest0-21swapper/309:45:173
1603899240,16cyclictest0-21swapper/308:25:133
1603899240,12cyclictest0-21swapper/308:20:193
1603199241,9cyclictest0-21swapper/209:47:112
1603199241,11cyclictest0-21swapper/210:43:372
1603199241,10cyclictest0-21swapper/211:57:592
1603199241,10cyclictest0-21swapper/210:17:312
1603199241,10cyclictest0-21swapper/210:08:112
1603199241,10cyclictest0-21swapper/207:36:072
1603199240,11cyclictest0-21swapper/210:55:252
1603199240,11cyclictest0-21swapper/207:15:472
1602599241,16cyclictest6197-21/usr/sbin/munin11:40:141
1602599241,13cyclictest2674150kworker/u8:011:30:381
1602599241,13cyclictest2674150kworker/u8:011:30:381
1602599240,12cyclictest10385-21kworker/u8:110:11:281
1601799243,9cyclictest81ktimersoftd/009:30:130
1601799243,10cyclictest0-21swapper/007:30:010
1601799241,10cyclictest25495-21kworker/0:108:03:110
1603899238,2cyclictest0-21swapper/311:27:513
1603899238,2cyclictest0-21swapper/311:21:153
1603899234,5cyclictest87350irq/126-enp1s0-07:47:423
1603899234,10cyclictest87350irq/126-enp1s0-09:16:403
1603899232,7cyclictest0-21swapper/311:06:363
1603899231,6cyclictest0-21swapper/310:11:303
1603899231,10cyclictest5328-21head07:55:203
1603899230,12cyclictest0-21swapper/312:25:203
1603899230,10cyclictest0-21swapper/307:55:013
1603199231,16cyclictest0-21swapper/209:21:372
1603199231,14cyclictest0-21swapper/209:15:182
1603199231,13cyclictest0-21swapper/209:10:212
1603199230,9cyclictest0-21swapper/209:41:232
1603199230,10cyclictest0-21swapper/208:10:212
1602599232,9cyclictest87550irq/128-enp1s0-10:55:131
1602599231,15cyclictest0-21swapper/111:50:191
1602599231,15cyclictest0-21swapper/109:53:321
1601799235,5cyclictest0-21swapper/011:18:080
1601799231,4cyclictest0-21swapper/009:55:100
162022221,2sleep00-21swapper/012:00:230
1603899228,3cyclictest0-21swapper/311:15:553
16038992212,7cyclictest25157-21ntpq08:40:193
16038992212,7cyclictest23890-21fschecks_count07:25:163
1603899221,12cyclictest6700-21cpuspeed_turbos08:00:123
1603899221,12cyclictest21925-21/usr/sbin/munin08:35:213
1603899221,12cyclictest20090-21forks08:30:143
1603899220,12cyclictest3574-21/usr/sbin/munin09:05:173
1603199221,14cyclictest0-21swapper/212:07:132
1603199221,14cyclictest0-21swapper/207:13:332
1603199220,9cyclictest0-21swapper/212:13:552
1603199220,9cyclictest0-21swapper/208:06:232
1603199220,9cyclictest0-21swapper/208:06:232
1603199220,8cyclictest0-21swapper/211:31:472
1603199220,8cyclictest0-21swapper/211:31:472
1603199220,15cyclictest0-21swapper/212:22:392
1602599228,12cyclictest10385-21kworker/u8:110:25:321
1602599227,4cyclictest0-21swapper/112:27:001
1602599223,2cyclictest87550irq/128-enp1s0-10:15:131
1602599221,18cyclictest16209-21rt-features12:00:221
1602599221,16cyclictest0-21swapper/111:48:111
1602599221,15cyclictest0-21swapper/109:42:031
1602599221,11cyclictest0-21swapper/111:35:281
1602599221,11cyclictest0-21swapper/110:54:361
1601799222,11cyclictest32304-21/usr/sbin/munin07:45:240
1603899219,3cyclictest21317-21/usr/sbin/munin07:20:153
1603899216,3cyclictest26934-21grep08:45:153
1603899213,12cyclictest0-21swapper/310:04:233
1603899212,11cyclictest87550irq/128-enp1s0-12:35:123
16038992116,3cyclictest30749-21cut07:40:163
16038992110,8cyclictest28004-21cpuspeed_turbos07:35:093
16038992110,7cyclictest10704-21latency_hist08:10:013
16038992110,7cyclictest10704-21latency_hist08:10:013
1603899210,13cyclictest87350irq/126-enp1s0-12:15:143
1603199211,14cyclictest0-21swapper/208:41:152
1603199211,11cyclictest0-21swapper/207:33:472
1603199211,10cyclictest0-21swapper/211:53:172
1603199211,10cyclictest0-21swapper/210:47:192
1602599219,3cyclictest14991-21head10:45:151
1602599212,11cyclictest19537-21grep12:10:121
1602599211,18cyclictest0-21swapper/111:20:171
1602599211,18cyclictest0-21swapper/109:47:441
1602599211,17cyclictest26882-21ps07:30:201
1602599211,15cyclictest0-21swapper/107:51:221
1602599211,15cyclictest0-21swapper/107:47:561
1602599211,14cyclictest0-21swapper/107:20:291
1602599210,9cyclictest0-21swapper/108:59:431
1602599210,16cyclictest0-21swapper/107:27:521
1602599210,15cyclictest87550irq/128-enp1s0-10:43:061
1602599210,15cyclictest0-21swapper/111:55:261
1601799212,10cyclictest29154-21/usr/sbin/munin10:05:140
1601799210,5cyclictest11-21rcuc/011:30:370
1601799210,5cyclictest11-21rcuc/011:30:370
1603899204,4cyclictest29192-21hddtemp_smartct08:50:153
1603899201,4cyclictest22958-21sleep11:05:013
1603199207,3cyclictest0-21swapper/209:07:112
1603199200,11cyclictest0-21swapper/208:01:352
1602599202,2cyclictest87550irq/128-enp1s0-12:07:331
1602599201,6cyclictest0-21swapper/111:18:101
16025992016,3cyclictest0-21swapper/108:52:041
16025992016,3cyclictest0-21swapper/108:25:461
1602599201,3cyclictest0-21swapper/109:35:421
1602599201,17cyclictest11297-21chrt09:24:481
1602599201,14cyclictest0-21swapper/108:08:061
1602599201,14cyclictest0-21swapper/108:08:061
1602599201,14cyclictest0-21swapper/107:39:491
1602599201,11cyclictest714-21cstates09:00:131
1602599201,11cyclictest32314-21diskstats11:25:131
1602599201,11cyclictest26054-21smartctl11:10:151
1602599201,11cyclictest25037-21memory08:40:191
1602599201,11cyclictest15331-21/usr/sbin/munin08:20:131
1602599201,11cyclictest14091-21sh09:30:141
1602599201,11cyclictest14046-21memory08:15:181
1602599201,11cyclictest0-21swapper/112:20:341
1602599201,11cyclictest0-21swapper/108:03:231
1602599201,10cyclictest31200-21open_files07:40:221
1602599201,10cyclictest21719-21wc08:35:011
1602599201,10cyclictest0-21swapper/111:00:371
1602599200,2cyclictest87550irq/128-enp1s0-10:20:161
1602599200,2cyclictest87550irq/128-enp1s0-10:20:161
1601799202,10cyclictest31353-21/usr/sbin/munin10:10:150
1601799202,10cyclictest1169-21/usr/sbin/munin10:15:180
1601799201,9cyclictest0-21swapper/012:37:000
1601799201,11cyclictest19399-21/usr/sbin/munin12:10:200
1601799201,11cyclictest17198-21/usr/sbin/munin12:05:170
16038991912,4cyclictest1860-21switchtime09:00:243
1603899191,11cyclictest0-21swapper/307:10:113
1603899190,11cyclictest0-21swapper/308:10:193
16025991916,2cyclictest0-21swapper/112:31:411
16025991912,4cyclictest30752-21fschecks_count12:35:151
1602599191,16cyclictest0-21swapper/108:35:201
16025991911,3cyclictest0-21swapper/107:59:031
1602599191,10cyclictest7746-21grep09:15:151
1602599190,5cyclictest0-21swapper/111:07:231
1602599190,14cyclictest0-21swapper/107:11:421
1601799199,2cyclictest808-21snmpd08:07:590
1601799199,2cyclictest808-21snmpd08:07:580
1601799198,3cyclictest30718-21/usr/sbin/munin08:55:160
1601799196,3cyclictest7-21ksoftirqd/011:05:010
1601799192,3cyclictest4685-21diskstats07:55:140
1601799192,14cyclictest23779-21find09:50:230
1601799191,6cyclictest9-21rcu_preempt10:49:450
1601799191,6cyclictest0-21swapper/011:39:550
1601799191,5cyclictest0-21swapper/009:10:570
1601799191,11cyclictest0-21swapper/009:08:010
1601799191,10cyclictest28184-21/usr/sbin/munin12:30:200
1601799191,10cyclictest23179-21users12:15:270
1601799191,10cyclictest16218-21sshd07:10:060
1601799190,12cyclictest0-21swapper/011:45:220
286272181,13sleep028856-21df08:50:130
1603899180,6cyclictest0-21swapper/307:30:173
1602599181,9cyclictest25775-21/usr/sbin/munin09:55:211
1602599181,9cyclictest0-21swapper/110:30:191
16025991812,3cyclictest10947-21acpi08:10:111
1602599181,13cyclictest87550irq/128-enp1s0-09:27:321
1602599180,16cyclictest0-21swapper/109:05:261
1601799186,3cyclictest22305-1kworker/u9:210:38:400
1601799185,5cyclictest6614-21ntp_kernel_err10:25:200
1601799183,9cyclictest0-21swapper/011:13:150
1601799183,6cyclictest27903-21/usr/sbin/munin07:35:140
1601799183,5cyclictest11-21rcuc/009:40:220
1601799181,9cyclictest24848-21ntpq12:20:220
1601799181,8cyclictest0-21swapper/012:26:530
1601799181,7cyclictest2674150kworker/u8:011:26:440
1601799181,5cyclictest774-21avahi-daemon08:33:480
1601799181,5cyclictest7-21ksoftirqd/011:55:130
1601799181,4cyclictest0-21swapper/008:46:410
1601799181,11cyclictest0-21swapper/011:52:550
1601799181,10cyclictest18224-21sshd10:51:050
139582181,2sleep00-21swapper/010:42:490
16025991711,3cyclictest19453-21df07:15:131
1601799179,5cyclictest0-21swapper/007:30:230
1601799174,4cyclictest31044-21switchtime11:20:230
1601799173,5cyclictest87550irq/128-enp1s0-09:15:120
1601799171,5cyclictest0-21swapper/011:43:470
1601799171,5cyclictest0-21swapper/010:20:050
1601799171,5cyclictest0-21swapper/010:20:050
1601799171,4cyclictest21308-21sshd07:20:060
1601799171,4cyclictest0-21swapper/009:25:440
1601799171,14cyclictest4731-21diskmemload09:36:510
1601799170,7cyclictest87550irq/128-enp1s0-09:20:190
1601799170,15cyclictest0-21swapper/008:40:140
1603899168,5cyclictest0-21swapper/308:15:113
1601799169,1cyclictest0-21swapper/008:20:210
1601799168,5cyclictest31760-21sshd07:41:060
1601799164,3cyclictest3183-21grep07:50:190
1601799161,7cyclictest0-21swapper/009:00:120
1601799161,7cyclictest0-21swapper/008:10:060
16017991612,3cyclictest0-21swapper/010:33:020
1601799161,14cyclictest0-21swapper/007:17:580
16017991611,3cyclictest14451-21cat08:15:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional