You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-23 - 17:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Thu May 23, 2024 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39447449912218,37cyclictest4003097-21unin-run21:55:010
3944744991032,11cyclictest880726-21firefox21:13:100
39447489910189,8cyclictest26-21ksoftirqd/122:05:101
394474899993,91cyclictest4057450-40lsb_release00:27:231
394474899993,91cyclictest4057450-40lsb_release00:27:231
162399980,77rtkit-daemon0-21swapper/119:07:511
3944744999688,5cyclictest0-21swapper/022:18:340
3944744999684,8cyclictest0-21swapper/021:31:440
3944744999684,8cyclictest0-21swapper/021:31:440
394474499962,90cyclictest0-21swapper/022:05:170
394474499962,90cyclictest0-21swapper/020:08:510
394458129656,33sleep20-21swapper/219:09:422
394474899952,89cyclictest0-21swapper/121:13:101
394474499952,89cyclictest0-21swapper/023:00:410
394474499952,89cyclictest0-21swapper/022:52:270
394474499952,7cyclictest0-21swapper/000:22:250
394474499952,6cyclictest0-21swapper/000:07:290
394474899942,88cyclictest0-21swapper/123:05:051
3944744999484,6cyclictest0-21swapper/021:23:370
394474499942,88cyclictest0-21swapper/022:42:000
394474499942,88cyclictest0-21swapper/021:45:550
394474499942,88cyclictest0-21swapper/020:14:460
394474499942,88cyclictest0-21swapper/000:27:020
394474499942,88cyclictest0-21swapper/000:27:020
394474499942,7cyclictest0-21swapper/023:11:530
394474499942,7cyclictest0-21swapper/023:11:520
394474499942,7cyclictest0-21swapper/022:57:490
394474499942,7cyclictest0-21swapper/019:33:550
3944748999385,5cyclictest0-21swapper/120:08:511
3944748999385,5cyclictest0-21swapper/100:31:561
394474899932,88cyclictest0-21swapper/122:15:201
394474899932,87cyclictest0-21swapper/121:35:201
394474899932,6cyclictest0-21swapper/122:10:281
394474899932,6cyclictest0-21swapper/122:00:141
3944744999385,5cyclictest0-21swapper/000:35:110
3944744999385,5cyclictest0-21swapper/000:31:060
3944744999384,6cyclictest0-21swapper/022:49:480
3944744999383,6cyclictest0-21swapper/021:42:580
3944744999382,8cyclictest0-21swapper/020:15:520
394474499933,87cyclictest0-21swapper/000:12:340
394474499932,87cyclictest0-21swapper/023:07:410
394474499932,87cyclictest0-21swapper/022:11:330
394474499932,87cyclictest0-21swapper/021:37:090
394474499932,87cyclictest0-21swapper/021:15:470
394474499932,87cyclictest0-21swapper/021:07:260
394474499932,87cyclictest0-21swapper/000:16:390
394474499932,7cyclictest0-21swapper/022:33:570
394474499932,6cyclictest0-21swapper/000:04:470
394474499932,6cyclictest0-21swapper/000:04:470
394437229372,13sleep00-21swapper/019:06:590
3944748999285,4cyclictest0-21swapper/122:53:371
3944748999285,4cyclictest0-21swapper/119:34:121
3944748999284,5cyclictest0-21swapper/123:00:411
394474899922,87cyclictest0-21swapper/121:27:451
3944744999284,5cyclictest0-21swapper/021:04:150
3944744999282,6cyclictest0-21swapper/022:00:380
394474499922,8cyclictest0-21swapper/020:03:500
394474499922,87cyclictest0-21swapper/021:27:270
394474499922,86cyclictest0-21swapper/022:35:490
3944748999183,5cyclictest0-21swapper/122:55:481
3944748999183,5cyclictest0-21swapper/122:42:001
3944748999183,5cyclictest0-21swapper/100:12:341
3944748999183,5cyclictest0-21swapper/100:06:181
394474899912,87cyclictest0-21swapper/121:07:491
394474899912,85cyclictest0-21swapper/120:12:191
3944744999183,5cyclictest0-21swapper/022:21:000
3944744999183,5cyclictest0-21swapper/022:21:000
3944744999181,7cyclictest0-21swapper/023:33:490
3944748999083,5cyclictest0-21swapper/121:47:001
3944748999083,4cyclictest0-21swapper/121:54:241
3944748999083,4cyclictest0-21swapper/100:16:401
3944748999082,5cyclictest0-21swapper/100:36:091
394474899902,86cyclictest0-21swapper/123:12:001
394474899902,86cyclictest0-21swapper/123:12:001
3944748998981,5cyclictest0-21swapper/120:16:271
394474899892,81cyclictest4003339-21cpu21:55:131
3944748998881,4cyclictest0-21swapper/121:18:251
3944748998880,5cyclictest0-21swapper/100:23:421
39748842870,5sleep00-21swapper/020:35:000
394475299874,15cyclictest141rcu_preempt22:51:262
3944748998779,5cyclictest0-21swapper/122:35:481
3944748998779,5cyclictest0-21swapper/122:34:431
3944748998779,5cyclictest0-21swapper/121:31:191
3944748998779,5cyclictest0-21swapper/121:31:191
3944748998678,5cyclictest0-21swapper/122:48:041
3944748998678,5cyclictest0-21swapper/121:41:361
3944748998678,5cyclictest0-21swapper/121:04:541
3944748998678,5cyclictest0-21swapper/120:04:431
394448328670,9sleep30-21swapper/319:08:273
3944752998525,18cyclictest34-21ksoftirqd/219:34:442
3944748998577,5cyclictest0-21swapper/123:15:071
3944756998363,8cyclictest1753-21gnome-shell23:07:283
3944756998322,25cyclictest42-21ksoftirqd/321:37:493
3944748998275,5cyclictest0-21swapper/100:04:321
3944748998275,5cyclictest0-21swapper/100:04:321
3944748998274,5cyclictest0-21swapper/122:20:291
3944748998274,5cyclictest0-21swapper/122:20:291
394474899822,37cyclictest880726-21firefox21:21:161
3944752998117,8cyclictest34-21ksoftirqd/223:00:472
394474499812,16cyclictest1753-21gnome-shell21:55:340
394475299809,6cyclictest141rcu_preempt21:10:162
3944748997971,5cyclictest0-21swapper/123:33:591
162399790,5rtkit-daemon0-21swapper/023:18:050
394475299772,8cyclictest141rcu_preempt20:15:192
394475699762,17cyclictest880726-21firefox00:11:193
394475299764,14cyclictest141rcu_preempt21:35:352
394475299763,15cyclictest141rcu_preempt21:20:142
394474899761,38cyclictest4033578-21expr23:20:141
394475299753,7cyclictest141rcu_preempt21:42:222
394475299752,8cyclictest141rcu_preempt23:10:582
394475299752,8cyclictest141rcu_preempt23:10:582
3944752997525,46cyclictest881251-21Softwar~cThread22:14:542
3944752997519,50cyclictest994-21lldpd22:37:132
394475299751,7cyclictest141rcu_preempt22:15:052
3944756997420,49cyclictest881251-21Softwar~cThread21:08:563
3944756997410,14cyclictest881251-21Softwar~cThread21:25:593
394475299745,7cyclictest141rcu_preempt00:07:572
3944752997452,7cyclictest34-21ksoftirqd/220:10:382
394475299744,7cyclictest141rcu_preempt22:03:142
394475699733,13cyclictest1753-21gnome-shell22:01:343
394475699732,30cyclictest1753-21gnome-shell20:09:273
162399730,5rtkit-daemon0-21swapper/023:29:050
394475699722,24cyclictest881253-21Renderer00:33:503
394475699713,64cyclictest1753-21gnome-shell23:11:203
394475699713,64cyclictest1753-21gnome-shell23:11:203
394475699712,10cyclictest1753-21gnome-shell00:05:223
3944756997117,13cyclictest42-21ksoftirqd/323:00:243
3944752997116,50cyclictest881251-21Softwar~cThread00:15:492
394475699703,62cyclictest1753-21gnome-shell21:41:463
394475699703,60cyclictest1753-21gnome-shell22:33:503
394475699703,46cyclictest1753-21gnome-shell21:45:253
394475699703,13cyclictest1753-21gnome-shell21:32:453
394475699703,13cyclictest1753-21gnome-shell21:32:443
394475699702,13cyclictest880726-21firefox21:19:173
3944756997015,49cyclictest881251-21Softwar~cThread00:24:293
394475299706,15cyclictest141rcu_preempt21:15:152
394475299701,7cyclictest141rcu_preempt21:05:332
3944752997011,9cyclictest1753-21gnome-shell23:15:232
3944756996950,14cyclictest42-21ksoftirqd/321:50:223
3944756996913,51cyclictest881251-21Softwar~cThread00:35:263
394475299692,7cyclictest0-21swapper/221:49:552
394475299692,61cyclictest34-21ksoftirqd/200:30:022
394475299692,61cyclictest34-21ksoftirqd/200:30:012
394475299692,60cyclictest1753-21gnome-shell20:03:502
3944756996857,7cyclictest1069-21runrttasks22:49:183
3944756996855,8cyclictest1753-21gnome-shell22:57:323
3944756996851,5cyclictest42-21ksoftirqd/322:40:193
3944756996840,23cyclictest1753-21gnome-shell23:33:533
3944752996814,48cyclictest881251-21Softwar~cThread22:59:212
394474899682,37cyclictest3972244-21memory20:25:251
3944756996754,9cyclictest1753-21gnome-shell20:19:523
3944756996751,5cyclictest42-21ksoftirqd/320:10:443
394475299677,9cyclictest1753-21gnome-shell22:33:502
3944752996749,6cyclictest34-21ksoftirqd/200:32:142
3944756996653,9cyclictest1753-21gnome-shell21:21:063
394475699663,10cyclictest1753-21gnome-shell22:10:293
3944756996615,47cyclictest1753-21gnome-shell00:29:103
3944756996615,47cyclictest1753-21gnome-shell00:29:103
394475299669,8cyclictest34-21ksoftirqd/222:06:322
3944752996649,12cyclictest881251-21Softwar~cThread00:24:312
3944756996553,8cyclictest1753-21gnome-shell21:58:483
394475699652,60cyclictest0-21swapper/322:54:513
394475699652,21cyclictest1753-21gnome-shell22:15:233
3944752996546,7cyclictest34-21ksoftirqd/221:25:162
394475299652,20cyclictest1753-21gnome-shell21:30:182
394475299652,20cyclictest1753-21gnome-shell21:30:182
3944752996515,45cyclictest34-21ksoftirqd/221:55:032
3944756996454,6cyclictest42-21ksoftirqd/322:37:533
394475699642,56cyclictest1753-21gnome-shell20:04:123
394475299642,13cyclictest1753-21gnome-shell00:13:512
3944748996416,43cyclictest3973689-21cat20:30:131
394474899641,38cyclictest4046527-21cat23:55:341
3944748996410,33cyclictest3959632-21cstates19:50:131
3944756996348,5cyclictest42-21ksoftirqd/300:18:173
394475699632,51cyclictest1753-21gnome-shell22:20:383
394475699632,51cyclictest1753-21gnome-shell22:20:373
3944752996352,8cyclictest1753-21gnome-shell22:44:212
394475299632,57cyclictest1753-21gnome-shell00:04:562
394475299632,57cyclictest1753-21gnome-shell00:04:552
394475299632,20cyclictest1753-21gnome-shell20:05:052
394474899632,48cyclictest1753-21gnome-shell20:20:031
3944756996250,7cyclictest880726-21firefox23:15:233
394475699622,19cyclictest1753-21gnome-shell22:08:023
394475299622,18cyclictest1753-21gnome-shell22:45:272
394475699612,16cyclictest880726-21firefox21:10:423
3944748996112,43cyclictest881251-21Softwar~cThread19:10:191
394475699603,51cyclictest1753-21gnome-shell00:03:503
394475699603,51cyclictest1753-21gnome-shell00:03:503
394475299603,10cyclictest1753-21gnome-shell00:35:262
394475299602,54cyclictest1753-21gnome-shell23:05:412
394475299602,54cyclictest1753-21gnome-shell21:50:552
394475299602,53cyclictest1753-21gnome-shell23:33:522
394475299602,53cyclictest1753-21gnome-shell19:35:082
394474899602,36cyclictest1753-21gnome-shell20:59:111
394474899602,33cyclictest3949306-21fw_packets19:20:201
394474899601,37cyclictest3948248-21cat19:15:391
162399600,9rtkit-daemon1622-21rtkit-daemon19:49:150
394475699592,53cyclictest1753-21gnome-shell19:34:053
3944748995912,41cyclictest881251-21Softwar~cThread19:26:351
394475699583,13cyclictest1753-21gnome-shell21:04:253
3944752995810,7cyclictest0-21swapper/222:21:222
3944752995810,7cyclictest0-21swapper/222:21:222
394474899589,45cyclictest4042462-21cat23:45:261
394474499582,51cyclictest1753-21gnome-shell20:20:090
394475299573,50cyclictest1753-21gnome-shell21:04:042
394474899571,35cyclictest3980040-21taskset20:49:551
394474499574,48cyclictest881356-21Isolated14:31:440
394475699562,48cyclictest1753-21gnome-shell19:35:113
394474899562,37cyclictest3981623-40lsb_release20:51:251
3944748995518,32cyclictest4038657-21grep23:35:131
3944748995518,32cyclictest4038657-21grep23:35:131
3944748995518,32cyclictest3956356-21cut19:40:221
394474899541,48cyclictest3955166-21/usr/sbin/munin19:35:351
162399540,8rtkit-daemon1622-21rtkit-daemon19:22:213
40362432520,7sleep342-21ksoftirqd/323:26:513
3944748995212,35cyclictest4014476-21if_enp2s022:25:201
3944744995220,28cyclictest1753-21gnome-shell20:59:110
162399520,8rtkit-daemon1622-21rtkit-daemon20:37:110
162399520,8rtkit-daemon1622-21rtkit-daemon19:26:333
394474899512,33cyclictest1753-21gnome-shell19:57:221
3944744995119,28cyclictest1069-21runrttasks23:55:110
394474899502,43cyclictest3957988-21df19:45:181
3944748995017,28cyclictest3975695-21latency20:35:281
3944748995014,32cyclictest3977319-21cat20:40:241
3944748995013,32cyclictest4043767-21cpu23:50:151
3944744995024,22cyclictest1753-21gnome-shell23:49:580
3944744995020,25cyclictest4041474-21sshd23:42:060
3944756994917,12cyclictest881251-21Softwar~cThread20:47:443
394474899491,44cyclictest4035688-21/usr/sbin/munin23:25:281
394474499491,5cyclictest0-21swapper/023:22:070
39830202480,20sleep342-21ksoftirqd/320:55:363
39774742480,18sleep342-21ksoftirqd/320:40:273
3944756994841,4cyclictest0-21swapper/320:20:033
394474899482,40cyclictest4040731-21/usr/sbin/munin23:40:251
394474499483,39cyclictest3956709-21grep19:40:310
394474499479,18cyclictest1753-21gnome-shell20:40:410
394474499479,18cyclictest1753-21gnome-shell19:20:460
394474499474,39cyclictest3981085-21needreboot20:50:260
3944744994714,29cyclictest3952112-21sh19:30:000
3944744994713,29cyclictest3960886-21mktemp19:55:020
394475299464,11cyclictest34-21ksoftirqd/220:55:192
3944752994615,14cyclictest141rcu_preempt19:55:282
394474499466,13cyclictest855-21dbus-daemon20:50:010
394474499463,38cyclictest1069-21runrttasks23:38:270
394474499463,38cyclictest1069-21runrttasks23:38:270
3944744994620,21cyclictest3961788-21missed_timers19:55:280
3944744994616,26cyclictest1069-21runrttasks22:27:190
3944744994615,26cyclictest4043933-21cstates23:50:160
3944744994613,29cyclictest3971628-21/usr/sbin/munin20:25:270
3944744994611,11cyclictest881251-21Softwar~cThread19:10:290
3944752994538,4cyclictest0-21swapper/220:20:062
394475699448,32cyclictest1753-21gnome-shell19:47:123
3944756994417,22cyclictest42-21ksoftirqd/319:15:193
3944752994414,8cyclictest438-20systemd-journal19:30:002
3944744994413,27cyclictest0-21swapper/019:20:000
3944756994324,7cyclictest42-21ksoftirqd/320:30:273
3944756994224,14cyclictest42-21ksoftirqd/323:40:183
394475699421,20cyclictest0-21swapper/320:35:593
3944752994223,14cyclictest3958025-21sh19:45:252
3944756994025,11cyclictest880726-21firefox00:00:013
394475699402,34cyclictest1753-21gnome-shell23:49:113
394475299409,5cyclictest141rcu_preempt22:30:012
3944752994024,12cyclictest854-21avahi-daemon00:00:032
3944752993927,8cyclictest3972871-21Classif~14:31:442
394475299392,33cyclictest1753-21gnome-shell20:48:042
394475299392,32cyclictest880726-21firefox23:42:062
3944752993922,12cyclictest438-20systemd-journal23:40:012
3944752993922,12cyclictest438-20systemd-journal23:40:012
3944752993913,6cyclictest141rcu_preempt19:40:232
394475299391,12cyclictest141rcu_preempt20:51:302
3944756993824,11cyclictest42-21ksoftirqd/322:25:163
3944756993818,16cyclictest4034360-21wc23:20:383
394475699381,14cyclictest4039872-21taskset23:38:503
394475699381,14cyclictest4039872-21taskset23:38:493
3944752993828,6cyclictest4042190-21cstates23:45:192
3944752993821,13cyclictest3949645-21ntpq19:20:292
3944752993821,12cyclictest3945675-21ntpq19:10:272
3944756993719,13cyclictest881331-21Privileged14:31:443
3944756993719,13cyclictest1632-21dbus-daemon19:44:123
3944752993721,12cyclictest3947571-21awk19:15:222
3944752993720,12cyclictest881251-21Softwar~cThread23:24:362
394475699362,8cyclictest1753-21gnome-shell20:54:203
394475699362,10cyclictest0-21swapper/320:25:133
3944756993619,13cyclictest1753-21gnome-shell19:51:023
3944756993523,8cyclictest1753-21gnome-shell23:51:473
3944756993518,12cyclictest881251-21Softwar~cThread19:58:263
394475299352,28cyclictest881251-21Softwar~cThread20:38:262
3944752993520,11cyclictest4044090-21sh23:50:182
3944752993519,12cyclictest3974267-21python320:30:322
3944752993519,11cyclictest3976857-21cat20:40:122
3944752993518,13cyclictest4035280-21expr23:25:152
394475299342,6cyclictest141rcu_preempt19:51:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional