You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 02:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Fri Nov 28, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
277721020,38sleep12778-21cat19:09:001
266929910137,10cyclictest12966-21grep21:44:053
266929910137,10cyclictest12966-21grep21:44:053
26689999736,13cyclictest0-21swapper/123:33:271
162652900,15sleep32669299cyclictest22:59:013
52752850,3sleep1201ktimersoftd/120:23:361
26686998535,44cyclictest19074-21fschecks_count19:43:540
26686998524,53cyclictest0-21swapper/023:43:270
2668699835,30cyclictest0-21swapper/018:55:050
2668699833,28cyclictest1369-21runrttasks23:47:310
26686998324,12cyclictest0-21swapper/022:58:290
2668699829,8cyclictest114550irq/46-eth019:17:500
2668699824,71cyclictest0-21swapper/022:38:500
2668699824,30cyclictest0-21swapper/020:56:220
26686998230,46cyclictest3471-21unixbench_multi19:09:120
26686998230,46cyclictest1369-21runrttasks21:20:290
26686998230,46cyclictest1369-21runrttasks20:10:150
2668699823,9cyclictest13357-21turbostat00:03:270
2668699823,71cyclictest17181-21cut00:08:540
2668699823,24cyclictest3163-21grep22:33:450
26686998228,47cyclictest1369-21runrttasks21:58:500
26686998224,51cyclictest0-21swapper/020:47:590
26686998130,45cyclictest14750-21grep00:03:520
26686998122,9cyclictest1369-21runrttasks22:29:050
2668699804,11cyclictest0-21swapper/020:23:520
26686998029,9cyclictest28120-21sh20:03:500
2668699794,11cyclictest0-21swapper/019:48:560
2668699793,31cyclictest0-21swapper/019:58:500
2668699793,31cyclictest0-21swapper/000:18:590
2668699793,29cyclictest1369-21runrttasks21:15:330
2668699793,28cyclictest0-21swapper/020:33:490
2668699793,11cyclictest0-21swapper/019:38:570
2668699793,10cyclictest8468-21grep22:43:500
26686997919,9cyclictest19905-21sshd00:14:020
2668699784,31cyclictest0-21swapper/021:48:500
2668699784,12cyclictest0-21swapper/021:28:540
26686997830,42cyclictest26833-21munin-node21:08:490
26686997826,8cyclictest15303-21timerwakeupswit19:34:120
26686997826,8cyclictest15303-21timerwakeupswit19:34:120
26686997822,11cyclictest0-21swapper/023:03:440
26686997812,14cyclictest0-21swapper/023:53:270
2668699774,66cyclictest0-21swapper/023:08:480
2668699774,66cyclictest0-21swapper/023:08:480
2668699773,31cyclictest0-21swapper/023:23:430
26686997724,11cyclictest0-21swapper/021:03:480
26686997715,13cyclictest0-21swapper/022:48:460
2668699765,12cyclictest0-21swapper/022:25:210
2668699765,12cyclictest0-21swapper/022:25:210
2668699764,24cyclictest0-21swapper/019:24:090
2668699764,13cyclictest0-21swapper/022:11:310
2668699764,10cyclictest0-21swapper/023:13:490
2668699763,30cyclictest0-21swapper/022:14:360
2668699763,29cyclictest0-21swapper/022:08:310
26686997628,42cyclictest22039-21munin-node20:59:010
26686997622,8cyclictest1618-21nscd21:39:060
26686997621,9cyclictest1369-21runrttasks23:21:070
26686997618,9cyclictest12145-21gltestperf19:28:550
2668699754,12cyclictest0-21swapper/023:35:050
2668699753,31cyclictest0-21swapper/020:20:350
2668699753,11cyclictest0-21swapper/018:59:020
26686997524,9cyclictest0-21swapper/021:33:470
26686997524,7cyclictest16407-21ls22:59:020
26686997523,10cyclictest0-21swapper/021:23:480
2668699745,9cyclictest0-21swapper/023:58:270
2668699745,9cyclictest0-21swapper/023:58:270
2668699743,11cyclictest0-21swapper/019:19:130
26686997425,10cyclictest0-21swapper/022:18:460
26686997420,10cyclictest659-21cpuspeed_turbos20:13:500
26686997420,10cyclictest659-21cpuspeed_turbos20:13:500
26686997418,12cyclictest0-21swapper/021:53:440
2668699734,13cyclictest0-21swapper/019:04:570
2668699724,11cyclictest0-21swapper/020:38:580
2668699723,30cyclictest0-21swapper/023:28:430
2668699723,14cyclictest0-21swapper/019:58:360
26686997229,9cyclictest11839-21fschecks_count21:43:500
26686997229,9cyclictest11839-21fschecks_count21:43:500
2668699703,10cyclictest0-21swapper/020:28:550
2668699693,15cyclictest0-21swapper/020:48:550
2524726310,43sleep00-21swapper/018:48:580
2524726310,43sleep00-21swapper/018:48:580
243799620,4rtkit-daemon2436-21rtkit-daemon20:14:312
243799620,4rtkit-daemon2436-21rtkit-daemon20:14:312
243799620,4rtkit-daemon2436-21rtkit-daemon00:09:442
243799600,4rtkit-daemon2436-21rtkit-daemon19:57:172
215592603,13sleep20-21swapper/222:03:512
83662593,11sleep10-21swapper/119:19:211
243799590,4rtkit-daemon2436-21rtkit-daemon22:48:402
198572599,8sleep119993-21expr20:53:491
2669299584,49cyclictest0-21swapper/321:28:333
2668999584,24cyclictest0-21swapper/122:43:431
26689995831,13cyclictest0-21swapper/120:28:471
26691995723,15cyclictest0-21swapper/221:58:322
2668999563,47cyclictest0-21swapper/122:23:491
2668999563,47cyclictest0-21swapper/122:23:491
26689995628,13cyclictest0-21swapper/123:01:481
26689995627,14cyclictest0-21swapper/120:03:501
243799560,5rtkit-daemon2436-21rtkit-daemon23:48:002
300722553,44sleep20-21swapper/223:28:512
2669199558,27cyclictest0-21swapper/223:18:402
26691995534,13cyclictest32537-21sed19:04:012
26691995524,13cyclictest0-21swapper/220:54:022
26691995523,14cyclictest0-21swapper/221:33:472
26691995522,14cyclictest0-21swapper/223:13:562
2668999557,28cyclictest0-21swapper/119:29:071
26689995529,12cyclictest0-21swapper/122:13:581
26689995527,13cyclictest0-21swapper/122:08:451
26689995526,15cyclictest0-21swapper/122:03:441
26689995523,16cyclictest0-21swapper/119:33:521
26689995523,16cyclictest0-21swapper/119:33:521
26689995520,16cyclictest0-21swapper/120:33:481
243799550,5rtkit-daemon2436-21rtkit-daemon19:48:102
243799550,5rtkit-daemon2436-21rtkit-daemon00:13:442
163962556,43sleep116507-21df_abs00:08:441
64772543,45sleep10-21swapper/121:33:471
2669299544,45cyclictest0-21swapper/320:28:363
2669299544,42cyclictest0-21swapper/322:28:443
2669199546,29cyclictest0-21swapper/223:33:572
2669199545,29cyclictest0-21swapper/221:14:102
2669199545,29cyclictest0-21swapper/220:19:012
2669199544,31cyclictest0-21swapper/221:28:542
2669199544,30cyclictest0-21swapper/220:39:092
2669199544,29cyclictest0-21swapper/220:28:572
2669199544,29cyclictest0-21swapper/219:43:372
2669199544,28cyclictest0-21swapper/222:13:502
26691995434,12cyclictest4937-21grep19:13:562
2669199543,36cyclictest0-21swapper/219:19:092
26691995422,14cyclictest0-21swapper/221:03:472
26691995420,14cyclictest0-21swapper/222:54:012
26691995416,20cyclictest0-21swapper/220:23:482
2668999543,45cyclictest0-21swapper/119:39:001
26689995427,14cyclictest0-21swapper/123:48:021
26689995423,15cyclictest0-21swapper/122:33:421
243799540,6rtkit-daemon2436-21rtkit-daemon22:44:032
243799540,5rtkit-daemon2436-21rtkit-daemon21:18:512
243799540,5rtkit-daemon2436-21rtkit-daemon21:04:073
243799540,5rtkit-daemon2436-21rtkit-daemon20:48:072
243799540,5rtkit-daemon2436-21rtkit-daemon20:09:012
243799540,5rtkit-daemon2436-21rtkit-daemon20:04:092
144742543,43sleep10-21swapper/100:03:491
26692995338,9cyclictest27510-21taskset18:53:573
26692995321,12cyclictest0-21swapper/323:53:273
2669199537,27cyclictest0-21swapper/221:48:332
2669199537,27cyclictest0-21swapper/221:48:332
2669199536,36cyclictest0-21swapper/220:33:502
2669199535,28cyclictest0-21swapper/223:58:402
2669199535,27cyclictest0-21swapper/219:00:052
2669199534,30cyclictest0-21swapper/223:53:592
2669199534,30cyclictest0-21swapper/223:53:592
2669199534,30cyclictest0-21swapper/221:58:442
2669199534,30cyclictest0-21swapper/219:23:522
2669199534,28cyclictest0-21swapper/222:18:522
2669199534,28cyclictest0-21swapper/220:48:482
26691995322,13cyclictest0-21swapper/223:38:382
26691995322,13cyclictest0-21swapper/222:23:432
26691995322,13cyclictest0-21swapper/222:23:432
26691995321,15cyclictest0-21swapper/223:23:412
26691995321,13cyclictest0-21swapper/222:10:382
26691995319,14cyclictest0-21swapper/221:38:562
2668999535,30cyclictest0-21swapper/121:58:481
2668999534,42cyclictest16341-21cpuspeed_turbos21:53:461
26689995328,10cyclictest27717-21gltestperf18:53:591
26689995327,12cyclictest0-21swapper/120:16:111
26689995327,12cyclictest0-21swapper/120:16:111
26689995326,13cyclictest0-21swapper/123:19:001
26689995325,15cyclictest0-21swapper/119:43:511
26689995325,14cyclictest0-21swapper/118:58:521
26689995324,15cyclictest0-21swapper/121:28:571
26689995323,9cyclictest11139-21cat21:43:331
26689995322,13cyclictest0-21swapper/119:05:061
243799530,5rtkit-daemon2436-21rtkit-daemon19:58:542
243799530,5rtkit-daemon2436-21rtkit-daemon19:34:032
243799530,5rtkit-daemon2436-21rtkit-daemon19:34:032
237772533,43sleep10-21swapper/123:14:111
155762533,11sleep10-21swapper/121:50:441
52892523,41sleep35364-21needreboot19:14:033
2669299524,43cyclictest0-21swapper/319:53:363
2669299524,42cyclictest30426-21aten2.4-expect22:23:433
2669299524,42cyclictest30426-21aten2.4-expect22:23:433
2669299523,44cyclictest0-21swapper/321:39:023
2669299523,44cyclictest0-21swapper/321:10:343
2669299523,42cyclictest29894-21fschecks_count18:58:573
26692995226,9cyclictest7824-21telnet20:28:483
26692995219,13cyclictest0-21swapper/323:18:573
2669199525,30cyclictest0-21swapper/223:08:462
2669199525,30cyclictest0-21swapper/223:08:462
2669199525,29cyclictest0-21swapper/221:23:502
2669199525,28cyclictest0-21swapper/222:28:482
2669199524,29cyclictest0-21swapper/222:39:022
2669199524,29cyclictest0-21swapper/219:29:002
2669199524,29cyclictest0-21swapper/218:54:062
2669199524,28cyclictest0-21swapper/221:10:282
2669199524,27cyclictest0-21swapper/221:49:002
2669199523,41cyclictest22310-21sshd19:49:072
26691995220,15cyclictest0-21swapper/220:58:512
2668999525,28cyclictest0-21swapper/122:33:311
2668999523,41cyclictest0-21swapper/123:48:381
26689995223,12cyclictest1369-21runrttasks23:06:361
26689995222,14cyclictest0-21swapper/119:48:521
243799520,5rtkit-daemon2436-21rtkit-daemon23:50:382
243799520,5rtkit-daemon2436-21rtkit-daemon00:20:592
243799520,5rtkit-daemon2436-21rtkit-daemon00:03:452
3602513,41sleep3419-21aten2.4-expect20:13:483
3602513,41sleep3419-21aten2.4-expect20:13:483
2669299514,42cyclictest0-21swapper/300:18:383
2669299513,40cyclictest0-21swapper/300:03:283
26692995124,8cyclictest14628-21awk20:43:343
26692995123,22cyclictest19602-21aten2.4-expect23:08:413
26692995123,22cyclictest19602-21aten2.4-expect23:08:413
2669299512,43cyclictest0-21swapper/319:24:063
2669199514,28cyclictest0-21swapper/223:03:472
2669199514,28cyclictest0-21swapper/223:00:472
2669199514,28cyclictest0-21swapper/222:33:512
2669199514,28cyclictest0-21swapper/219:13:382
2669199513,28cyclictest0-21swapper/218:53:392
2669199513,28cyclictest0-21swapper/218:53:392
2668999515,28cyclictest0-21swapper/123:33:571
2668999515,28cyclictest0-21swapper/123:23:411
2668999515,28cyclictest0-21swapper/121:03:091
2668999515,28cyclictest0-21swapper/120:44:061
2668999515,28cyclictest0-21swapper/119:28:081
2668999514,29cyclictest0-21swapper/121:09:111
2668999514,28cyclictest0-21swapper/121:18:331
2668999513,41cyclictest5409-21aten2.4-expect20:23:481
26689995124,13cyclictest0-21swapper/122:55:461
26689995120,13cyclictest0-21swapper/121:47:581
26689995120,13cyclictest0-21swapper/121:47:581
26689995115,10cyclictest23834-21nscd22:53:291
243799510,5rtkit-daemon2436-21rtkit-daemon20:34:003
240402512,9sleep30-21swapper/323:17:323
186392512,7sleep318700-21sed23:04:013
157412512,9sleep30-21swapper/319:37:123
157412512,9sleep30-21swapper/319:37:123
135762515,9sleep30-21swapper/300:03:303
2669299504,41cyclictest0-21swapper/321:48:563
2669299504,41cyclictest0-21swapper/300:13:433
2669299503,42cyclictest0-21swapper/320:51:203
2668999505,26cyclictest0-21swapper/121:20:301
2668999505,24cyclictest19901-21ls00:14:021
2668999504,39cyclictest0-21swapper/121:06:101
26689995032,9cyclictest11532-21cpuspeed_turbos23:58:431
26689995025,11cyclictest0-21swapper/122:18:571
243799500,5rtkit-daemon2436-21rtkit-daemon22:02:273
190702504,39sleep10-21swapper/120:49:131
2669299494,40cyclictest0-21swapper/323:55:373
2669299494,40cyclictest0-21swapper/323:55:373
2669299493,42cyclictest0-21swapper/320:59:303
2669299493,41cyclictest0-21swapper/320:08:163
2668999493,40cyclictest0-21swapper/120:08:491
95882482,9sleep30-21swapper/322:45:563
34872483,40sleep10-21swapper/123:39:421
322642482,9sleep30-21swapper/320:09:253
276412484,10sleep30-21swapper/322:17:143
2669299484,39cyclictest0-21swapper/321:23:283
2669299484,39cyclictest0-21swapper/319:19:483
26692994832,10cyclictest29563-21cut21:13:483
2669299483,41cyclictest0-21swapper/323:38:023
2669299483,41cyclictest0-21swapper/319:53:583
2669299483,41cyclictest0-21swapper/319:12:373
2669299483,40cyclictest0-21swapper/321:53:443
26692994824,8cyclictest25662-21munin-node19:59:013
26692994821,12cyclictest14860-21munin-node20:44:133
26692994820,12cyclictest0-21swapper/319:33:373
26689994829,13cyclictest5618-21sshd19:14:091
26689994824,9cyclictest23998-21fschecks_time19:53:551
106312487,10sleep10-21swapper/123:54:091
106312487,10sleep10-21swapper/123:54:091
2669299474,39cyclictest0-21swapper/300:12:393
2669299474,37cyclictest0-21swapper/320:54:083
2669299474,37cyclictest0-21swapper/320:18:583
26692994731,8cyclictest13039-21grep22:53:483
2669299473,40cyclictest0-21swapper/321:29:533
2669299473,40cyclictest0-21swapper/319:43:233
2669299473,39cyclictest0-21swapper/323:23:413
2669299473,38cyclictest0-21swapper/319:44:073
2669299472,40cyclictest0-21swapper/322:10:103
26692994711,27cyclictest1628-21nscd23:48:273
26689994732,8cyclictest20407-21head23:08:491
26689994732,8cyclictest20407-21head23:08:491
2669299464,12cyclictest0-21swapper/322:03:433
2669299463,24cyclictest31682-21munin-node19:03:563
2668999464,23cyclictest0-21swapper/118:53:401
2668999464,23cyclictest0-21swapper/118:53:401
243799460,4rtkit-daemon2436-21rtkit-daemon22:33:483
145412463,37sleep10-21swapper/120:42:401
119422462,9sleep30-21swapper/322:50:273
71312453,35sleep10-21swapper/122:40:131
2669299453,35cyclictest0-21swapper/323:31:103
2669299453,26cyclictest1876-21cpuspeed_turbos23:38:423
26692994517,12cyclictest0-21swapper/322:38:433
26692994515,24cyclictest6479-21aten_r1power_cu21:33:463
26689994527,12cyclictest2549-21gltestperf21:23:531
2668999443,36cyclictest0-21swapper/100:19:381
26692994329,8cyclictest29586-21sshd22:19:053
26689994316,8cyclictest114550irq/46-eth019:58:541
243799360,4rtkit-daemon2436-21rtkit-daemon18:49:423
243799360,4rtkit-daemon2436-21rtkit-daemon18:49:423
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional