You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 07:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Fri May 09, 2025 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
227799930,8rtkit-daemon0-21swapper/316:51:123
227799930,8rtkit-daemon0-21swapper/316:51:123
25492997217,14cyclictest0-21swapper/321:06:063
2548899699,37cyclictest0-21swapper/016:55:580
25488996821,40cyclictest10640-21tune2fs21:55:500
25488996616,43cyclictest0-21swapper/017:30:270
25488996544,14cyclictest22433-21grep21:11:060
227799650,5rtkit-daemon2276-21rtkit-daemon17:00:573
25488996417,39cyclictest19670-21munin-node17:45:570
25488996416,39cyclictest16442-21chrt19:51:060
25488996414,43cyclictest0-21swapper/020:10:320
25488996413,44cyclictest0-21swapper/018:00:380
25488996322,35cyclictest20351-21grep22:15:490
25488996310,45cyclictest0-21swapper/019:35:590
25488996310,45cyclictest0-21swapper/019:35:590
98812624,50sleep30-21swapper/317:25:393
2548899629,35cyclictest0-21swapper/019:10:470
25488996211,44cyclictest0-21swapper/017:15:390
274422613,49sleep30-21swapper/320:15:533
274422613,49sleep30-21swapper/320:15:533
25488996118,37cyclictest17196-21taskset19:55:460
25488996115,29cyclictest0-21swapper/021:36:010
25488996014,27cyclictest12273-21munin-node22:00:500
25488996012,41cyclictest0-21swapper/020:26:070
25488996011,31cyclictest0-21swapper/020:05:320
2549299593,50cyclictest0-21swapper/320:01:013
2549099594,47cyclictest31162-21idleruntime21:30:491
25488995918,35cyclictest32061-21sed19:16:080
25488995918,34cyclictest18329-21sendmail18:50:360
25488995917,28cyclictest11232-21tail20:50:430
25488995915,18cyclictest0-21swapper/021:50:530
25488995914,38cyclictest0-21swapper/018:42:540
25488995914,27cyclictest25590-21munin-node19:06:070
227799590,4rtkit-daemon2276-21rtkit-daemon20:42:413
25488995817,26cyclictest17269-21munin-node17:41:040
25488995816,35cyclictest17129-21basename22:10:400
25488995813,37cyclictest0-21swapper/018:30:470
25488995813,29cyclictest0-21swapper/021:25:290
25488995812,40cyclictest0-21swapper/017:51:110
25488995812,39cyclictest0-21swapper/017:21:110
25488995812,31cyclictest0-21swapper/017:30:490
25488995812,31cyclictest0-21swapper/017:30:490
25488995812,30cyclictest23058-21munin-node21:15:470
227799580,4rtkit-daemon2276-21rtkit-daemon22:13:373
227799580,3rtkit-daemon2276-21rtkit-daemon19:26:212
83132572,17sleep20-21swapper/218:26:372
25492995741,10cyclictest28959-21gltestperf16:56:003
25492995738,12cyclictest13153-21cut19:45:543
25488995717,26cyclictest0-21swapper/016:50:520
25488995717,26cyclictest0-21swapper/016:50:520
25488995713,36cyclictest6922-21grep19:31:110
25488995713,26cyclictest30474-21munin-node17:00:530
25488995713,24cyclictest26475-21latency_hist20:15:330
25488995712,38cyclictest0-21swapper/019:26:020
25488995712,29cyclictest0-21swapper/022:05:470
2548899571,50cyclictest0-21swapper/021:01:000
2548899571,50cyclictest0-21swapper/021:01:000
133932572,47sleep30-21swapper/318:40:363
2549299568,41cyclictest0-21swapper/320:55:473
2549299565,27cyclictest11020-21munin-run18:35:353
2549199564,45cyclictest0-21swapper/219:55:332
2549199563,45cyclictest0-21swapper/221:30:462
2549099564,30cyclictest0-21swapper/119:15:461
25488995615,34cyclictest29989-21latency_hist21:30:290
25488995613,36cyclictest0-21swapper/021:05:570
227799560,5rtkit-daemon2276-21rtkit-daemon21:15:472
205682563,7sleep320645-21sed18:55:363
2549299555,30cyclictest22961-21munin-run19:00:353
2549299555,30cyclictest22961-21munin-run19:00:353
2549099554,29cyclictest0-21swapper/117:11:131
2549099554,27cyclictest0-21swapper/117:56:071
2548899558,31cyclictest0-21swapper/019:41:090
2548899552,44cyclictest0-21swapper/021:46:010
2548899552,44cyclictest0-21swapper/021:46:010
25488995514,34cyclictest28857-21latency_hist20:20:330
25488995514,34cyclictest28857-21latency_hist20:20:330
25488995514,23cyclictest0-21swapper/020:50:300
25488995513,37cyclictest0-21swapper/020:25:330
25488995513,37cyclictest0-21swapper/020:25:330
227799550,5rtkit-daemon2276-21rtkit-daemon19:06:522
227799550,5rtkit-daemon2276-21rtkit-daemon19:01:002
227799550,5rtkit-daemon2276-21rtkit-daemon17:06:493
2549299546,28cyclictest0-21swapper/322:00:483
25492995432,16cyclictest26140-21sed17:56:103
25492995421,13cyclictest0-21swapper/317:50:503
2549199543,44cyclictest0-21swapper/221:20:522
25490995425,12cyclictest0-21swapper/118:41:041
25490995422,14cyclictest0-21swapper/119:11:071
2548899547,31cyclictest0-21swapper/020:35:440
2548899545,26cyclictest0-21swapper/020:55:410
2548899544,43cyclictest0-21swapper/018:20:540
25488995430,18cyclictest1263-21runrttasks21:30:350
25488995414,33cyclictest0-21swapper/019:01:060
25488995412,26cyclictest292151munin-node18:06:130
25488995410,37cyclictest0-21swapper/017:16:360
227799540,5rtkit-daemon2276-21rtkit-daemon19:20:592
227799540,5rtkit-daemon2276-21rtkit-daemon19:10:522
227799540,5rtkit-daemon2276-21rtkit-daemon18:31:082
227799540,5rtkit-daemon2276-21rtkit-daemon18:15:542
227799540,5rtkit-daemon2276-21rtkit-daemon17:57:172
31602533,12sleep20-21swapper/217:10:542
2549299535,29cyclictest0-21swapper/322:15:553
2549299535,29cyclictest0-21swapper/322:05:483
2549299535,29cyclictest0-21swapper/319:40:473
2549299535,28cyclictest0-21swapper/321:11:063
2549299534,30cyclictest0-21swapper/317:45:563
2549299534,28cyclictest0-21swapper/320:25:323
2549299534,28cyclictest0-21swapper/320:25:323
25492995321,14cyclictest0-21swapper/319:15:533
2549099535,30cyclictest0-21swapper/122:00:401
2549099535,30cyclictest0-21swapper/121:15:461
2549099534,29cyclictest0-21swapper/120:30:331
2548899538,38cyclictest0-21swapper/018:10:500
2548899538,38cyclictest0-21swapper/018:10:500
2548899535,41cyclictest0-21swapper/017:35:500
25488995330,17cyclictest1263-21runrttasks18:38:030
25488995316,15cyclictest0-21swapper/017:06:030
233782533,43sleep10-21swapper/117:51:081
227799530,6rtkit-daemon2276-21rtkit-daemon21:31:013
227799530,5rtkit-daemon2276-21rtkit-daemon20:45:483
227799530,5rtkit-daemon2276-21rtkit-daemon18:46:082
227799530,5rtkit-daemon2276-21rtkit-daemon17:21:042
227799530,4rtkit-daemon2276-21rtkit-daemon21:36:013
227799530,4rtkit-daemon2276-21rtkit-daemon19:35:512
227799530,4rtkit-daemon2276-21rtkit-daemon19:35:512
227799530,4rtkit-daemon2276-21rtkit-daemon17:41:182
2549299525,28cyclictest0-21swapper/319:35:443
2549299525,28cyclictest0-21swapper/319:35:443
2549299525,28cyclictest0-21swapper/318:41:023
2549299525,28cyclictest0-21swapper/318:01:083
2549299525,28cyclictest0-21swapper/317:25:513
2549299525,28cyclictest0-21swapper/317:16:133
2549299525,27cyclictest17738-21df_inode19:55:513
2549299524,29cyclictest0-21swapper/318:05:563
2549299524,29cyclictest0-21swapper/317:34:073
2549299524,29cyclictest0-21swapper/317:34:073
2549299524,28cyclictest0-21swapper/319:23:313
2549299524,28cyclictest0-21swapper/318:11:023
2549299524,28cyclictest0-21swapper/318:11:023
25492995236,10cyclictest8621-21crond18:30:363
25492995232,12cyclictest23746-21cut21:15:493
25492995222,13cyclictest0-21swapper/320:36:003
25492995222,12cyclictest0-21swapper/319:50:443
25492995218,13cyclictest0-21swapper/317:35:503
2549199523,43cyclictest0-21swapper/220:45:582
2549199523,42cyclictest0-21swapper/221:05:522
25491995229,10cyclictest1138-21snmpd17:34:502
25491995229,10cyclictest1138-21snmpd17:34:502
2549099524,40cyclictest0-21swapper/117:50:391
2549099524,30cyclictest0-21swapper/122:05:481
2549099524,30cyclictest0-21swapper/121:35:501
2549099524,30cyclictest0-21swapper/120:31:011
2549099524,30cyclictest0-21swapper/119:36:031
2549099524,30cyclictest0-21swapper/119:36:031
2549099524,30cyclictest0-21swapper/118:46:081
2549099524,30cyclictest0-21swapper/118:21:091
2549099524,29cyclictest0-21swapper/120:15:331
2549099524,26cyclictest0-21swapper/118:15:371
2549099524,26cyclictest0-21swapper/118:15:371
2549099523,44cyclictest0-21swapper/121:06:031
2548899525,41cyclictest0-21swapper/018:54:300
2548899524,40cyclictest0-21swapper/019:45:460
25488995233,13cyclictest22476-21unixbench-2d18:56:080
25488995233,13cyclictest22476-21unixbench-2d18:56:080
25488995229,18cyclictest1263-21runrttasks18:20:260
25488995212,25cyclictest6375-21aten2.4-expect20:40:420
227799520,5rtkit-daemon2276-21rtkit-daemon21:27:392
227799520,5rtkit-daemon2276-21rtkit-daemon19:56:052
227799520,5rtkit-daemon2276-21rtkit-daemon18:56:072
227799520,5rtkit-daemon2276-21rtkit-daemon18:56:072
227799520,5rtkit-daemon2276-21rtkit-daemon18:40:532
227799520,5rtkit-daemon2276-21rtkit-daemon18:05:542
220652522,42sleep10-21swapper/120:05:471
2549299515,28cyclictest0-21swapper/319:31:123
2549299514,28cyclictest0-21swapper/321:01:023
2549299514,28cyclictest0-21swapper/321:01:023
2549299514,28cyclictest0-21swapper/319:02:113
2549299514,27cyclictest0-21swapper/321:50:293
2549299514,27cyclictest0-21swapper/321:50:293
2549299514,27cyclictest0-21swapper/319:10:363
25492995120,13cyclictest0-21swapper/320:05:583
25492995119,13cyclictest0-21swapper/317:40:543
25492995116,14cyclictest0-21swapper/318:20:583
2549199515,39cyclictest0-21swapper/220:25:502
25491995133,11cyclictest15001-21cpuspeed_turbos22:05:432
25491995131,12cyclictest8708-21tail21:50:562
2549199513,40cyclictest0-21swapper/217:35:562
25491995125,21cyclictest0-21swapper/220:15:322
2549199512,43cyclictest0-21swapper/221:55:512
2549099515,28cyclictest0-21swapper/121:11:031
2549099515,28cyclictest0-21swapper/117:36:041
2549099514,30cyclictest0-21swapper/118:51:111
25490995129,10cyclictest0-21swapper/119:00:571
25488995127,17cyclictest1263-21runrttasks20:33:560
227799510,5rtkit-daemon2276-21rtkit-daemon18:02:542
227799510,5rtkit-daemon2276-21rtkit-daemon17:18:412
227799510,4rtkit-daemon2276-21rtkit-daemon19:16:072
132382512,9sleep20-21swapper/220:54:582
2549299505,27cyclictest0-21swapper/320:50:533
2549299503,28cyclictest0-21swapper/321:50:463
25492995020,12cyclictest0-21swapper/319:12:523
25492995019,10cyclictest3744-21iostat_ios21:40:553
25492995013,31cyclictest36402sleep318:17:503
2549099504,38cyclictest0-21swapper/117:40:561
2549099504,29cyclictest0-21swapper/120:45:321
2549099504,29cyclictest0-21swapper/116:50:411
2549099504,27cyclictest0-21swapper/119:06:091
25490995033,10cyclictest108482sleep119:40:551
25490995021,25cyclictest0-21swapper/116:53:211
25490995021,25cyclictest0-21swapper/116:53:211
25490995017,11cyclictest3002-21telnet19:25:481
2548899508,36cyclictest0-21swapper/021:40:470
2548899507,37cyclictest0-21swapper/018:25:530
227799500,4rtkit-daemon2276-21rtkit-daemon21:55:533
227799500,4rtkit-daemon2276-21rtkit-daemon17:26:032
214072503,40sleep20-21swapper/220:01:492
198532502,9sleep20-21swapper/218:51:072
323042492,9sleep10-21swapper/117:01:261
313582492,9sleep10-21swapper/118:10:311
2549299494,25cyclictest0-21swapper/320:31:033
2549299493,28cyclictest0-21swapper/318:46:113
2549299493,26cyclictest0-21swapper/320:25:453
2549299493,23cyclictest0-21swapper/319:25:593
25492994913,13cyclictest0-21swapper/321:20:423
2549199494,38cyclictest0-21swapper/220:35:482
25491994923,19cyclictest15665-21crond21:00:302
2549199492,42cyclictest0-21swapper/218:35:502
2549199492,42cyclictest0-21swapper/218:11:042
2549199492,42cyclictest0-21swapper/218:11:042
2549199492,41cyclictest0-21swapper/219:30:482
2549099495,28cyclictest0-21swapper/117:16:141
2549099494,29cyclictest0-21swapper/121:50:291
2549099494,29cyclictest0-21swapper/121:50:291
2549099494,29cyclictest0-21swapper/120:00:541
2549099494,29cyclictest0-21swapper/118:55:541
2549099494,29cyclictest0-21swapper/118:55:541
2549099493,14cyclictest0-21swapper/117:06:071
25490994920,14cyclictest0-21swapper/119:35:341
25488994930,12cyclictest1263-21runrttasks18:03:340
227799490,4rtkit-daemon2276-21rtkit-daemon19:45:582
155362492,41sleep10-21swapper/120:58:521
275482482,9sleep10-21swapper/121:24:561
2549299485,14cyclictest0-21swapper/320:10:513
2549299484,23cyclictest27930-21taskset21:25:423
2549199484,38cyclictest0-21swapper/221:45:452
2549199484,38cyclictest0-21swapper/221:45:452
2549199483,38cyclictest0-21swapper/220:20:502
2549199483,38cyclictest0-21swapper/220:20:502
2549199482,41cyclictest0-21swapper/221:00:482
2549199482,41cyclictest0-21swapper/221:00:482
2549099484,37cyclictest0-21swapper/119:50:501
2549099484,28cyclictest0-21swapper/121:51:001
2549099484,28cyclictest0-21swapper/117:25:591
2549099484,11cyclictest28868-21awk20:20:321
2549099484,11cyclictest28868-21awk20:20:321
2549099483,39cyclictest4611-21fschecks_count20:35:501
25490994821,12cyclictest0-21swapper/119:45:481
2548899482,18cyclictest0-21swapper/019:20:540
227799480,4rtkit-daemon2276-21rtkit-daemon16:51:422
227799480,4rtkit-daemon2276-21rtkit-daemon16:51:422
131482482,9sleep10-21swapper/120:53:481
96982472,9sleep10-21swapper/118:30:551
46842472,9sleep20-21swapper/221:42:542
2549199474,37cyclictest0-21swapper/219:41:032
2549199474,36cyclictest0-21swapper/218:20:482
2549199473,15cyclictest0-21swapper/220:15:452
2549199473,15cyclictest0-21swapper/220:15:452
25491994716,18cyclictest6445-21telnet20:40:432
2549099475,35cyclictest0-21swapper/122:11:041
2549099475,28cyclictest0-21swapper/117:30:501
2549099475,28cyclictest0-21swapper/117:30:501
2549099475,14cyclictest0-21swapper/121:25:541
2549099474,29cyclictest0-21swapper/118:40:361
2549099474,26cyclictest0-21swapper/118:00:551
25490994725,15cyclictest0-21swapper/122:15:401
25490994718,13cyclictest0-21swapper/121:55:561
2548899475,17cyclictest0-21swapper/016:50:410
105642472,9sleep10-21swapper/120:46:101
2549299464,35cyclictest0-21swapper/317:11:133
25491994631,10cyclictest2049-21head21:36:042
2549199463,39cyclictest0-21swapper/217:01:082
25491994625,14cyclictest0-21swapper/216:50:402
2549199462,38cyclictest0-21swapper/217:50:512
2549199462,21cyclictest0-21swapper/222:17:142
2549099464,23cyclictest0-21swapper/121:00:561
2549099464,23cyclictest0-21swapper/121:00:561
2549099464,14cyclictest0-21swapper/116:56:071
25490994614,12cyclictest0-21swapper/117:21:021
48792452,37sleep10-21swapper/121:45:271
25491994520,10cyclictest20738-21telnet21:10:442
2549199452,38cyclictest0-21swapper/222:00:502
2549099454,26cyclictest0-21swapper/118:26:131
25490994530,8cyclictest19320-21munin-run20:00:321
2549099453,38cyclictest0-21swapper/119:23:461
2549099453,25cyclictest0-21swapper/120:21:001
2549099453,25cyclictest0-21swapper/120:21:001
24812452,9sleep20-21swapper/217:07:232
15712452,9sleep10-21swapper/118:15:431
14542442,9sleep20-21swapper/220:30:432
2549199433,12cyclictest0-21swapper/217:45:502
2549199423,13cyclictest0-21swapper/222:11:002
2549199422,34cyclictest0-21swapper/216:55:562
25491994216,21cyclictest24083-21latency_hist20:10:332
2549299363,19cyclictest255222sed16:50:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional