You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 18:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon May 04, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
116602235162,24sleep00-21swapper/007:09:560
115762234164,57sleep30-21swapper/307:08:493
116272202165,24sleep20-21swapper/207:09:292
113272199164,23sleep10-21swapper/107:05:341
82121450,3sleep31176899cyclictest07:55:163
1176299103100,2cyclictest0-21swapper/209:08:142
2078921020,5sleep01175299cyclictest09:55:130
11762997471,2cyclictest0-21swapper/208:23:492
44912640,5sleep24495-21fschecks_count11:45:152
44912640,5sleep24495-21fschecks_count11:45:142
306152560,4sleep121-21rcuc/111:30:171
303132490,1sleep10-21swapper/110:15:201
158352480,2sleep30-21swapper/307:15:203
1176299391,13cyclictest0-21swapper/209:12:222
11752993812,12cyclictest31861-21munin-run11:35:010
11762993734,2cyclictest0-21swapper/210:53:082
6172360,2sleep30-21swapper/311:35:213
1176299362,17cyclictest15711-21perl12:10:162
1176299341,25cyclictest0-21swapper/209:51:252
1176299341,22cyclictest0-21swapper/208:37:262
1176299341,22cyclictest0-21swapper/208:33:262
1176299341,21cyclictest8264-21needreboot09:25:192
1176299341,21cyclictest8264-21needreboot09:25:192
1176299341,18cyclictest0-21swapper/210:29:082
1175299345,14cyclictest5486-21munin-run10:35:000
1176299331,22cyclictest0-21swapper/207:29:082
1176299331,16cyclictest0-21swapper/208:04:172
1176299331,14cyclictest0-21swapper/210:08:282
1176299331,14cyclictest0-21swapper/207:42:402
1176299331,13cyclictest0-21swapper/209:38:162
1175299335,13cyclictest23079-21munin-run11:15:020
1175299334,14cyclictest16480-21munin-run11:00:020
1176299321,22cyclictest0-21swapper/209:33:012
1176299321,21cyclictest0-21swapper/207:39:122
1176299321,20cyclictest0-21swapper/210:45:042
1176299321,19cyclictest0-21swapper/212:27:322
1176299311,22cyclictest0-21swapper/211:28:072
1176299311,21cyclictest0-21swapper/208:43:452
1176299311,15cyclictest0-21swapper/211:38:312
1176299311,13cyclictest0-21swapper/211:30:032
1176299311,12cyclictest0-21swapper/211:10:042
1176299310,10cyclictest0-21swapper/211:58:502
11762993018,3cyclictest0-21swapper/208:47:342
1176299301,23cyclictest0-21swapper/209:23:422
1176299301,20cyclictest0-21swapper/212:39:522
1176299301,20cyclictest0-21swapper/208:29:492
1176299301,20cyclictest0-21swapper/208:16:292
1176299301,20cyclictest0-21swapper/208:16:292
1176299301,20cyclictest0-21swapper/208:07:492
1176299301,20cyclictest0-21swapper/207:21:012
1176299301,19cyclictest0-21swapper/208:51:132
1176299301,13cyclictest0-21swapper/210:24:122
1176299301,13cyclictest0-21swapper/210:11:562
1176299301,12cyclictest12934-21/usr/sbin/munin12:05:202
1176299301,12cyclictest0-21swapper/212:00:442
1176299301,11cyclictest0-21swapper/207:57:362
11768992919,2cyclictest0-21swapper/307:10:243
1176299291,19cyclictest0-21swapper/208:58:042
1176299291,19cyclictest0-21swapper/207:52:282
1176299291,19cyclictest0-21swapper/207:32:202
1176299291,19cyclictest0-21swapper/207:13:452
1176299291,13cyclictest0-21swapper/210:15:482
11762992911,3cyclictest0-21swapper/209:41:412
1176299291,12cyclictest0-21swapper/210:58:122
1176299291,12cyclictest0-21swapper/210:01:132
1176299291,12cyclictest0-21swapper/209:48:002
1176299291,12cyclictest0-21swapper/209:15:452
1176299291,12cyclictest0-21swapper/207:49:082
1176299291,11cyclictest0-21swapper/212:23:522
1176299291,11cyclictest0-21swapper/211:40:432
1176299291,11cyclictest0-21swapper/211:06:162
11762992910,3cyclictest0-21swapper/212:30:442
1175299296,10cyclictest15249-21perf08:30:000
1175299291,13cyclictest1070-21munin-run10:25:000
1176899281,12cyclictest0-21swapper/308:25:573
1176299282,8cyclictest17966-21smart_sda11:00:252
1176299281,18cyclictest0-21swapper/211:24:072
1176299281,12cyclictest0-21swapper/211:52:202
1176299281,12cyclictest0-21swapper/211:19:312
1176299281,12cyclictest0-21swapper/210:34:482
1176299281,10cyclictest0-21swapper/209:00:152
1176899278,3cyclictest0-21swapper/311:18:283
1176899275,12cyclictest87350irq/126-enp1s0-10:45:243
1176899271,10cyclictest20896-21/usr/sbin/munin08:40:223
1176899270,14cyclictest0-21swapper/310:10:193
11762992715,3cyclictest0-21swapper/207:17:012
1176299271,12cyclictest0-21swapper/210:41:122
1176299271,11cyclictest0-21swapper/208:12:172
1176299271,10cyclictest0-21swapper/209:58:142
1176299261,18cyclictest0-21swapper/212:17:042
11757992612,12cyclictest24447-21kworker/u8:207:20:341
1176899257,5cyclictest23961-21kworker/3:210:45:003
1176899253,12cyclictest87550irq/128-enp1s0-10:30:123
1176299251,10cyclictest0-21swapper/210:35:402
1176299251,10cyclictest0-21swapper/210:35:402
1175799251,20cyclictest6100-21munin-run11:50:011
1175799251,20cyclictest6100-21munin-run11:50:011
49332240,3sleep09-21rcu_preempt09:20:010
181922240,1sleep00-21swapper/008:35:140
1176899248,3cyclictest0-21swapper/308:02:283
1176899247,3cyclictest31240-21latency_hist10:20:013
1176899241,10cyclictest0-21swapper/311:30:263
1176899240,8cyclictest0-21swapper/307:50:243
11757992417,4cyclictest19300-21ls12:20:021
1175799241,17cyclictest0-21swapper/110:24:241
1175799241,12cyclictest0-21swapper/111:03:151
11757992410,7cyclictest827-21runrttasks07:14:011
1176899237,9cyclictest0-21swapper/307:21:573
1176899234,8cyclictest87550irq/128-enp1s0-12:02:073
1176899231,10cyclictest0-21swapper/311:09:143
1175799234,4cyclictest773-21NetworkManager07:16:471
1175799231,4cyclictest0-21swapper/111:55:141
1175799231,19cyclictest10394-21cat10:45:131
1175799231,16cyclictest0-21swapper/112:14:001
1175799231,16cyclictest0-21swapper/110:56:211
1175799231,16cyclictest0-21swapper/110:52:291
1175799231,11cyclictest0-21swapper/110:08:361
11757992310,11cyclictest32578-21kworker/u8:209:36:281
1175799230,16cyclictest16444-21kworker/u8:211:18:331
1175299238,5cyclictest10864-21tr12:00:110
1176899226,3cyclictest0-21swapper/312:30:263
1176899223,12cyclictest87650irq/129-enp1s0-07:43:163
1176899222,11cyclictest22637-21memory12:25:203
1176899221,13cyclictest87350irq/126-enp1s0-09:18:413
1176899221,12cyclictest87350irq/126-enp1s0-12:35:323
1175799226,13cyclictest25860-21systemd-journal08:50:271
1175799222,18cyclictest10064-21processes08:15:231
1175799222,18cyclictest10064-21processes08:15:231
1175799221,10cyclictest0-21swapper/109:30:281
1175299226,14cyclictest32213-21perf07:55:010
1175299221,6cyclictest0-21swapper/010:17:270
1176899219,3cyclictest5186-21ntp_states11:45:203
1176899219,3cyclictest5186-21ntp_states11:45:203
1176899216,4cyclictest0-21swapper/310:22:453
1176899215,9cyclictest0-21swapper/307:32:373
1176899215,4cyclictest0-21swapper/311:20:023
1176899213,7cyclictest0-21swapper/308:05:203
1176899213,3cyclictest87350irq/126-enp1s0-09:10:223
1176899211,11cyclictest17806-40dnf11:00:253
1176899210,6cyclictest0-21swapper/309:32:543
1175799219,3cyclictest20175-21smart_sda11:05:251
1175799218,11cyclictest18105-21kworker/u8:007:31:531
1175799213,13cyclictest87550irq/128-enp1s0-09:29:451
1175799213,13cyclictest87550irq/128-enp1s0-09:29:451
1175799212,9cyclictest2682750kworker/u8:110:26:141
1175799212,10cyclictest20050-21irqstats07:25:171
11757992117,3cyclictest0-21swapper/109:05:321
1175799211,5cyclictest32578-21kworker/u8:210:11:331
11757992114,4cyclictest26094-21gltestperf11:20:151
1175799211,3cyclictest5109-21wc09:20:111
1175799211,18cyclictest14681-21chrt12:07:171
1175799211,12cyclictest0-21swapper/109:55:431
1175799211,11cyclictest4548-21grep08:05:131
1175799211,11cyclictest2614-21diskstats08:00:141
1175799211,11cyclictest22403-21diskstats08:45:161
1175799211,11cyclictest22360-21grep12:25:171
1175799211,11cyclictest18572-21memory08:35:191
1175799211,11cyclictest0-21swapper/112:01:161
1175799210,9cyclictest0-21swapper/108:12:471
1175299212,3cyclictest26827-21kworker/u8:111:00:150
1175299212,11cyclictest28031-21/usr/sbin/munin07:45:270
1175299212,11cyclictest23303-21/usr/sbin/munin11:15:270
1175299211,8cyclictest28134-21ntp_offset10:10:200
161222200,1sleep00-21swapper/007:15:250
1176899208,6cyclictest87350irq/126-enp1s0-09:25:183
1176899208,6cyclictest87350irq/126-enp1s0-09:25:183
1176899208,3cyclictest13216-21memory10:50:193
1176899205,5cyclictest0-21swapper/311:44:413
1176899205,2cyclictest0-21swapper/309:02:073
1176899204,7cyclictest0-21swapper/311:26:103
1176899202,3cyclictest87350irq/126-enp1s0-12:11:313
1176899202,10cyclictest17658-21/usr/sbin/munin08:35:183
1176899201,6cyclictest0-21swapper/309:44:133
1176899201,2cyclictest87550irq/128-enp1s0-11:11:563
11768992011,3cyclictest0-21swapper/308:47:183
1176899201,12cyclictest0-21swapper/312:21:343
1176899201,12cyclictest0-21swapper/309:55:593
1176899201,10cyclictest87550irq/128-enp1s0-07:27:413
1176899200,9cyclictest0-21swapper/310:05:043
1176899200,4cyclictest16533-21sh08:30:213
1176899200,3cyclictest0-21swapper/308:20:133
1176899200,10cyclictest11676-21kworker/u8:109:39:543
1175799207,3cyclictest0-21swapper/109:40:181
1175799206,3cyclictest24447-21kworker/u8:208:24:291
1175799205,6cyclictest0-21swapper/110:30:471
1175799201,9cyclictest0-21swapper/109:46:401
11757992014,4cyclictest21370-21grep08:40:271
1175799201,17cyclictest0-21swapper/108:56:371
11757992011,3cyclictest87550irq/128-enp1s0-12:36:491
1175799201,12cyclictest0-21swapper/112:32:101
1175799201,12cyclictest0-21swapper/109:15:131
1175799201,11cyclictest21441-21df11:10:121
1175799201,10cyclictest8904-21/usr/sbin/munin10:40:201
1175299206,8cyclictest1896-40dnf07:58:250
1175299203,2cyclictest89250irq/135-enp2s0-11:20:320
11752992016,3cyclictest7-21ksoftirqd/009:30:010
11752992016,3cyclictest7-21ksoftirqd/009:30:010
1175299201,13cyclictest0-21swapper/009:35:260
1176899197,1cyclictest0-21swapper/307:45:043
1176899196,4cyclictest0-21swapper/312:08:443
1176899195,3cyclictest0-21swapper/308:19:073
1176899195,3cyclictest0-21swapper/308:19:073
1176899195,3cyclictest0-21swapper/308:11:233
1176899195,3cyclictest0-21swapper/307:35:233
1176899194,6cyclictest87350irq/126-enp1s0-10:35:163
1176899194,6cyclictest87350irq/126-enp1s0-10:35:163
1176899192,3cyclictest0-21swapper/311:55:433
1176899191,3cyclictest25336-21/usr/sbin/munin08:50:203
1176899191,10cyclictest14502-21/usr/sbin/munin10:55:273
1176899190,12cyclictest0-21swapper/312:16:543
1175799198,3cyclictest30577-21dnf07:50:141
1175799193,4cyclictest18898-21expr09:50:171
1175799192,6cyclictest0-21swapper/107:58:251
11757991915,3cyclictest0-21swapper/110:35:311
11757991915,3cyclictest0-21swapper/110:35:301
1175799191,17cyclictest0-21swapper/111:25:261
1175799191,17cyclictest0-21swapper/108:28:591
1175799191,11cyclictest0-21swapper/112:22:021
1175799191,11cyclictest0-21swapper/109:14:431
1175799191,10cyclictest7826-21seq11:50:271
1175799191,10cyclictest30508-21seq09:03:351
1175799191,10cyclictest2196-21expr11:40:121
1175799190,18cyclictest0-21swapper/107:42:501
1175299195,4cyclictest363-21diskmemload10:50:230
1175299191,7cyclictest0-21swapper/009:47:120
1175299191,7cyclictest0-21swapper/009:32:280
1175299191,6cyclictest0-21swapper/011:42:020
1175299191,5cyclictest0-21swapper/012:38:590
1175299191,5cyclictest0-21swapper/010:41:550
11752991914,3cyclictest8661-21grep08:15:010
11752991914,3cyclictest7-21ksoftirqd/011:50:180
11752991914,3cyclictest28424-21awk09:00:010
11752991913,3cyclictest20275-21ntpq07:25:200
1175299191,10cyclictest18399-21ntp_states12:15:220
1175299191,10cyclictest1636-21sshd11:39:240
238922181,2sleep00-21swapper/008:48:270
1176899185,6cyclictest0-21swapper/309:48:003
1176899184,3cyclictest87350irq/126-enp1s0-09:05:133
1176899183,3cyclictest0-21swapper/310:03:003
1176899182,2cyclictest87550irq/128-enp1s0-10:27:483
11768991812,3cyclictest26780-21df08:55:123
11768991811,4cyclictest5087-21/usr/sbin/munin09:20:123
1175799181,9cyclictest22446-40dnf10:00:021
11757991814,2cyclictest0-21swapper/107:35:171
1175799181,16cyclictest0-21swapper/108:31:031
1175799181,10cyclictest29536-21kworker/u8:111:36:421
1175799181,10cyclictest0-21swapper/107:45:151
1175299187,3cyclictest104502dnf10:45:150
1175299186,3cyclictest87350irq/126-enp1s0-10:35:230
1175299186,3cyclictest87350irq/126-enp1s0-10:35:230
1175299182,15cyclictest0-21swapper/008:42:290
1175299181,8cyclictest0-21swapper/011:06:240
1175299181,6cyclictest7919-1kworker/u9:011:57:260
1175299181,6cyclictest0-21swapper/012:08:280
1175299181,5cyclictest0-21swapper/012:25:410
1175299181,5cyclictest0-21swapper/009:40:550
1175299181,4cyclictest909-21sssd_nss12:30:150
1175299181,4cyclictest0-21swapper/011:47:570
1175299181,4cyclictest0-21swapper/011:47:570
1175299181,4cyclictest0-21swapper/010:09:120
276172171,2sleep00-21swapper/007:42:470
190452170,2sleep30-21swapper/309:50:183
1176899171,6cyclictest0-21swapper/311:54:053
1175299173,4cyclictest7-21ksoftirqd/011:25:150
1175299171,7cyclictest839-21lldpd07:30:100
1175299171,7cyclictest0-21swapper/007:35:140
1175299171,6cyclictest0-21swapper/008:00:270
1175299171,4cyclictest11060-21/usr/sbin/munin08:20:220
11752991713,3cyclictest0-21swapper/009:10:090
11752991713,2cyclictest7-21ksoftirqd/010:30:000
1175299171,2cyclictest190012sleep007:25:000
11752991711,3cyclictest8858-21seq08:15:080
11752991711,3cyclictest8858-21seq08:15:070
11752991711,3cyclictest29296-21gltestperf09:00:150
11752991711,3cyclictest25102-21expr08:50:150
1175299171,10cyclictest0-21swapper/012:11:360
1175299171,10cyclictest0-21swapper/009:20:030
11752991710,6cyclictest7-21ksoftirqd/007:10:500
1175299169,4cyclictest32119-21smart_sda09:05:250
1175299161,6cyclictest827-21runrttasks12:20:030
1175299161,6cyclictest0-21swapper/010:00:030
1175299161,6cyclictest0-21swapper/009:52:040
1175299161,14cyclictest0-21swapper/008:06:420
11752991611,3cyclictest827-21runrttasks08:34:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional