You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-13 - 03:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Fri Mar 13, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21972183108,25sleep10-21swapper/119:09:371
21162179101,23sleep20-21swapper/219:08:332
21592177107,25sleep00-21swapper/019:09:070
14662176123,42sleep30-21swapper/319:05:243
3197721740,4sleep2232099cyclictest22:40:222
2327995955,3cyclictest0-21swapper/319:24:313
167532560,3sleep20-21swapper/219:40:152
122592550,1sleep30-21swapper/321:55:243
95022500,4sleep1231499cyclictest21:50:161
149382480,4sleep339-21rcuc/322:01:583
317972450,1sleep30-21swapper/322:40:193
2327994025,3cyclictest0-21swapper/320:00:413
231499402,18cyclictest10972-21kworker/1:023:00:271
232799391,29cyclictest0-21swapper/322:30:493
232799391,29cyclictest0-21swapper/300:04:133
2327993817,13cyclictest0-21swapper/322:18:153
232799381,27cyclictest0-21swapper/319:58:493
2320993813,3cyclictest9217-21awk23:05:012
2306993810,13cyclictest10784-21munin-run21:55:010
232799371,27cyclictest0-21swapper/323:18:333
232799371,25cyclictest0-21swapper/322:35:293
232799371,25cyclictest0-21swapper/322:11:353
2327993712,3cyclictest496-21jbd2/dm-0-819:46:053
232799371,22cyclictest0-21swapper/319:27:293
230699378,14cyclictest12037-21munin-run00:25:010
232799361,22cyclictest0-21swapper/320:37:373
232799361,20cyclictest0-21swapper/300:17:513
232799351,28cyclictest0-21swapper/321:27:553
232799351,27cyclictest0-21swapper/322:05:513
232799351,23cyclictest0-21swapper/322:21:333
232799351,21cyclictest0-21swapper/320:09:033
232799351,18cyclictest0-21swapper/320:14:463
230699351,17cyclictest0-21swapper/021:09:110
304642340,4sleep20-21swapper/220:10:172
232799342,22cyclictest823-21snmpd23:31:533
232799341,28cyclictest0-21swapper/323:47:533
232799341,20cyclictest0-21swapper/323:25:533
232799341,20cyclictest0-21swapper/323:25:533
232799341,20cyclictest0-21swapper/320:46:473
230699341,15cyclictest0-21swapper/020:10:530
232799331,30cyclictest0-21swapper/319:50:533
232799331,30cyclictest0-21swapper/319:50:533
232799331,25cyclictest0-21swapper/323:51:353
232799331,25cyclictest0-21swapper/300:08:313
232799331,23cyclictest0-21swapper/322:57:513
232799331,20cyclictest0-21swapper/323:59:333
232799331,20cyclictest0-21swapper/321:47:053
232799331,20cyclictest0-21swapper/321:22:433
232799331,19cyclictest0-21swapper/323:35:553
232799331,18cyclictest0-21swapper/321:44:533
232799331,18cyclictest0-21swapper/321:00:513
232799331,18cyclictest0-21swapper/321:00:513
232799331,17cyclictest0-21swapper/319:17:453
232799331,16cyclictest0-21swapper/322:48:573
232799331,14cyclictest0-21swapper/321:05:113
232799331,12cyclictest0-21swapper/300:28:493
2320993311,3cyclictest27765-21awk21:20:012
232099330,5cyclictest2325-21kworker/u8:123:55:202
230699335,13cyclictest20189-21munin-run23:30:010
230699335,13cyclictest20189-21munin-run23:30:010
230699331,18cyclictest0-21swapper/020:21:170
230699331,10cyclictest0-21swapper/021:02:550
230699331,10cyclictest0-21swapper/021:02:550
232799321,20cyclictest0-21swapper/321:31:393
232799321,20cyclictest0-21swapper/300:37:573
232799321,20cyclictest0-21swapper/300:22:453
232799321,17cyclictest0-21swapper/320:53:313
232799321,15cyclictest0-21swapper/319:31:233
232799321,14cyclictest0-21swapper/321:54:393
232799321,14cyclictest0-21swapper/320:55:153
232799321,12cyclictest0-21swapper/322:28:553
232799321,12cyclictest0-21swapper/320:30:253
230699321,7cyclictest0-21swapper/022:15:310
230699321,11cyclictest0-21swapper/023:31:290
2327993116,7cyclictest0-21swapper/319:44:013
232799311,22cyclictest0-21swapper/300:33:493
232799311,17cyclictest0-21swapper/319:39:153
232799311,15cyclictest0-21swapper/321:10:033
232799311,14cyclictest0-21swapper/321:35:593
232799311,13cyclictest0-21swapper/320:44:413
232799311,12cyclictest0-21swapper/320:18:153
232799311,11cyclictest0-21swapper/320:23:333
230699311,7cyclictest0-21swapper/019:51:190
230699311,7cyclictest0-21swapper/019:51:190
230699311,18cyclictest0-21swapper/019:56:530
230699311,16cyclictest0-21swapper/023:50:210
230699311,11cyclictest0-21swapper/021:59:290
230699311,10cyclictest0-21swapper/021:24:430
232799301,20cyclictest0-21swapper/321:16:353
232799301,14cyclictest0-21swapper/323:11:393
232799301,13cyclictest0-21swapper/320:28:513
232799301,12cyclictest0-21swapper/300:10:033
230699301,18cyclictest0-21swapper/020:16:010
230699301,14cyclictest0-21swapper/019:25:110
230699301,11cyclictest0-21swapper/020:09:130
232799291,19cyclictest0-21swapper/323:22:213
232799291,14cyclictest0-21swapper/322:52:133
231499292,12cyclictest10972-21kworker/1:022:07:311
230699291,18cyclictest0-21swapper/020:48:470
230699291,13cyclictest0-21swapper/021:49:510
230699291,11cyclictest0-21swapper/022:27:130
230699291,11cyclictest0-21swapper/000:00:350
232799281,9cyclictest0-21swapper/323:06:433
2327992814,3cyclictest0-21swapper/319:14:353
232799281,17cyclictest0-21swapper/323:44:093
232799281,15cyclictest0-21swapper/323:03:573
232099283,3cyclictest89850irq/133-enp2s0-22:49:342
2320992816,3cyclictest0-21swapper/200:12:252
231499281,11cyclictest10972-21kworker/1:023:58:491
231499280,11cyclictest10972-21kworker/1:022:50:231
230699281,12cyclictest0-21swapper/021:39:130
230699281,12cyclictest0-21swapper/021:16:430
231499272,12cyclictest10972-21kworker/1:022:40:231
230699271,13cyclictest0-21swapper/020:52:070
230699271,13cyclictest0-21swapper/019:46:050
230699271,11cyclictest0-21swapper/019:38:030
232099261,17cyclictest0-21swapper/223:30:232
232099261,13cyclictest0-21swapper/220:38:442
2320992610,3cyclictest0-21swapper/221:41:482
231499261,17cyclictest0-21swapper/121:38:221
231499261,12cyclictest10972-21kworker/1:022:13:391
230699261,14cyclictest0-21swapper/023:10:270
230699261,14cyclictest0-21swapper/020:32:450
230699261,14cyclictest0-21swapper/019:22:130
230699261,11cyclictest0-21swapper/020:56:590
232099258,3cyclictest113352sleep219:25:262
2320992513,8cyclictest0-21swapper/223:49:342
232099251,11cyclictest0-21swapper/200:05:262
231499252,11cyclictest10972-21kworker/1:023:32:071
231499252,11cyclictest10972-21kworker/1:000:03:231
231499251,9cyclictest10972-21kworker/1:021:19:131
231499251,11cyclictest10972-21kworker/1:022:47:031
230699252,14cyclictest17691-21chrt23:21:210
230699252,11cyclictest31850-21ntp_states22:40:200
230699251,12cyclictest0-21swapper/023:35:330
230699251,12cyclictest0-21swapper/022:02:330
230699251,11cyclictest0-21swapper/019:17:210
232099249,3cyclictest0-21swapper/220:09:392
232099248,3cyclictest0-21swapper/220:54:222
232099248,3cyclictest0-21swapper/219:39:362
232099248,3cyclictest0-21swapper/219:12:282
2320992415,3cyclictest24654-21ntp_states21:10:222
2320992412,8cyclictest0-21swapper/221:54:442
231499249,2cyclictest10972-21kworker/1:020:25:131
231499242,11cyclictest10972-21kworker/1:019:46:571
231499241,11cyclictest10972-21kworker/1:021:24:551
231499241,10cyclictest10972-21kworker/1:022:55:371
230699247,14cyclictest27166-21idleruntime-cro20:05:010
230699241,16cyclictest0-21swapper/019:31:570
230699241,15cyclictest0-21swapper/000:11:070
230699241,14cyclictest0-21swapper/019:11:590
230699241,13cyclictest0-21swapper/022:49:210
230699241,13cyclictest0-21swapper/000:18:470
230699241,12cyclictest0-21swapper/023:42:590
230699241,12cyclictest0-21swapper/023:09:230
230699241,12cyclictest0-21swapper/022:09:210
230699241,12cyclictest0-21swapper/020:27:510
230699241,12cyclictest0-21swapper/000:39:590
230699241,11cyclictest0-21swapper/000:32:470
230699241,10cyclictest0-21swapper/000:28:110
230699240,3cyclictest0-21swapper/021:34:520
232099238,3cyclictest11642-21cpu23:10:112
232099238,3cyclictest0-21swapper/223:42:042
232099237,3cyclictest5731-21runrttasks20:29:282
232099237,3cyclictest0-21swapper/223:19:482
232099235,3cyclictest22476-40dnf21:05:222
232099235,3cyclictest0-21swapper/221:45:042
232099232,13cyclictest88150irq/126-enp1s0-21:27:272
232099231,6cyclictest0-21swapper/223:35:342
232099231,13cyclictest0-21swapper/200:30:132
232099231,11cyclictest0-21swapper/220:15:232
231499232,9cyclictest10972-21kworker/1:022:25:371
231499231,7cyclictest0-21swapper/123:22:511
231499231,11cyclictest10612-21irqstats19:25:181
231499230,9cyclictest0-21swapper/123:09:051
230699236,8cyclictest7221-21/usr/sbin/munin23:00:270
230699233,7cyclictest15109-21ls23:15:240
230699231,17cyclictest0-21swapper/022:58:500
230699231,15cyclictest0-21swapper/022:24:590
230699231,14cyclictest0-21swapper/021:11:450
230699231,12cyclictest0-21swapper/023:55:390
230699231,11cyclictest0-21swapper/019:43:570
230699231,10cyclictest0-21swapper/020:43:150
230699230,16cyclictest0-21swapper/021:27:210
63142220,4sleep232-21ksoftirqd/219:15:192
232099229,3cyclictest0-21swapper/220:02:062
232099228,8cyclictest0-21swapper/221:21:482
232099228,3cyclictest0-21swapper/222:17:102
232099228,3cyclictest0-21swapper/221:59:442
232099227,7cyclictest88350irq/128-enp1s0-20:47:172
232099227,3cyclictest0-21swapper/219:31:442
232099226,3cyclictest0-21swapper/220:34:202
232099223,5cyclictest7726-21cpuspeed_turbos19:20:102
232099223,5cyclictest28069-21sshd22:31:222
232099221,4cyclictest0-21swapper/200:17:572
2320992213,3cyclictest29256-21grep23:50:122
232099221,11cyclictest0-21swapper/222:25:252
232099220,12cyclictest0-21swapper/221:35:242
231499229,2cyclictest10972-21kworker/1:020:05:231
231499222,10cyclictest7380-21munin-run19:20:011
231499221,8cyclictest0-21swapper/123:11:051
2314992210,3cyclictest1951-21df_inode20:20:151
2314992210,3cyclictest15141-21diskstats20:50:141
231499220,9cyclictest0-21swapper/122:38:001
230699221,13cyclictest0-21swapper/022:30:550
230699221,12cyclictest0-21swapper/022:35:490
230699221,12cyclictest0-21swapper/021:42:290
230699221,11cyclictest0-21swapper/022:11:250
230699221,10cyclictest0-21swapper/022:53:090
230699221,10cyclictest0-21swapper/020:37:550
232099219,3cyclictest0-21swapper/222:10:562
232099218,3cyclictest0-21swapper/219:45:122
232099217,3cyclictest0-21swapper/222:57:502
232099216,4cyclictest0-21swapper/221:30:422
232099216,3cyclictest0-21swapper/222:51:502
232099216,3cyclictest0-21swapper/219:50:172
232099216,3cyclictest0-21swapper/219:50:172
231499214,4cyclictest10972-21kworker/1:020:35:331
231499212,9cyclictest10972-21kworker/1:020:41:471
231499212,12cyclictest10972-21kworker/1:020:49:391
231499212,11cyclictest10972-21kworker/1:020:13:191
231499212,10cyclictest10972-21kworker/1:019:53:131
231499212,10cyclictest10972-21kworker/1:019:53:131
231499211,8cyclictest0-21swapper/100:36:451
2314992115,3cyclictest11165-21cut21:55:131
2314992113,4cyclictest6393-21munin-run21:45:011
231499211,18cyclictest10972-21kworker/1:021:00:271
231499211,18cyclictest10972-21kworker/1:021:00:271
231499211,12cyclictest10972-21kworker/1:020:59:211
231499211,11cyclictest4522-21ntp_states00:05:221
231499211,10cyclictest27131-21cat20:04:561
231499211,10cyclictest21393-21/usr/sbin/munin21:05:121
230699211,12cyclictest0-21swapper/023:48:450
230699211,12cyclictest0-21swapper/000:08:340
232099209,3cyclictest0-21swapper/223:09:292
232099209,3cyclictest0-21swapper/222:22:202
232099209,3cyclictest0-21swapper/200:38:342
232099208,3cyclictest0-21swapper/200:01:482
232099207,6cyclictest0-21swapper/222:37:542
232099207,3cyclictest0-21swapper/223:29:452
232099207,3cyclictest0-21swapper/223:29:452
232099207,3cyclictest0-21swapper/220:43:422
232099206,3cyclictest0-21swapper/219:58:082
232099201,5cyclictest0-21swapper/200:21:572
232099201,4cyclictest0-21swapper/223:22:182
2320992013,4cyclictest19350-21cpuspeed_turbos21:00:122
2320992013,4cyclictest19350-21cpuspeed_turbos21:00:122
231499207,3cyclictest15600-21wc19:35:241
231499202,11cyclictest10972-21kworker/1:023:49:451
231499202,11cyclictest10972-21kworker/1:021:45:111
231499202,10cyclictest10972-21kworker/1:019:30:211
231499201,5cyclictest10972-21kworker/1:019:13:071
2314992014,3cyclictest17199-21irqstats19:40:181
231499201,13cyclictest10972-21kworker/1:000:28:271
231499201,10cyclictest24114-21awk19:55:231
231499201,10cyclictest19877-21sort22:15:111
231499201,10cyclictest1392-21cat20:20:011
231499201,10cyclictest10972-21kworker/1:020:31:171
231499201,10cyclictest0-21swapper/123:35:191
231499200,5cyclictest0-21swapper/121:33:401
231499200,11cyclictest0-21swapper/100:14:341
232099193,6cyclictest913-21sssd_nss20:55:192
232099191,17cyclictest0-21swapper/222:06:342
231499198,2cyclictest30182-21/usr/sbin/munin21:25:171
231499192,12cyclictest10972-21kworker/1:023:20:011
231499192,10cyclictest10972-21kworker/1:022:20:131
231499191,8cyclictest0-21swapper/123:40:571
231499191,3cyclictest19024-21irqstats23:25:191
231499191,3cyclictest19024-21irqstats23:25:191
231499191,3cyclictest10428-21cut00:20:141
2314991913,3cyclictest15129-21hddtemp_smartct00:30:171
231499191,11cyclictest10972-21kworker/1:021:11:531
231499190,9cyclictest0-21swapper/100:15:221
232099183,3cyclictest0-21swapper/220:21:222
231499186,3cyclictest31373-21cat23:55:021
231499181,9cyclictest0-21swapper/122:30:031
231499181,5cyclictest13203-21/usr/sbin/munin22:00:121
231499181,10cyclictest0-21swapper/119:20:191
232099171,7cyclictest0-21swapper/200:27:232
232099171,2cyclictest88250irq/127-enp1s0-22:00:452
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional