You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-07 - 22:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2s.osadl.org (updated Thu Dec 07, 2023 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4478210399,0sleep20-21swapper/207:01:082
291350650,0irq/25-eth10-21swapper/307:01:223
1075524240,0sleep00-21swapper/007:04:340
1081323026,0sleep10-21swapper/107:05:151
10930992020,0cyclictest27732-21kworker/1:110:15:491
10930991919,0cyclictest27732-21kworker/1:110:17:431
10930991919,0cyclictest27732-21kworker/1:109:55:031
10930991818,0cyclictest27732-21kworker/1:112:31:241
10930991717,0cyclictest27732-21kworker/1:108:29:211
10930991616,0cyclictest27732-21kworker/1:112:28:541
10930991616,0cyclictest27732-21kworker/1:112:02:281
10930991616,0cyclictest27732-21kworker/1:111:07:021
10930991616,0cyclictest27732-21kworker/1:110:40:001
10930991616,0cyclictest27732-21kworker/1:108:50:371
10930991616,0cyclictest27732-21kworker/1:108:08:351
10930991515,0cyclictest27732-21kworker/1:112:07:341
10930991515,0cyclictest27732-21kworker/1:111:22:441
10930991515,0cyclictest27732-21kworker/1:109:32:491
10930991515,0cyclictest27732-21kworker/1:109:06:031
10930991414,0cyclictest27732-21kworker/1:111:40:501
10930991414,0cyclictest27732-21kworker/1:109:21:111
10930991414,0cyclictest27732-21kworker/1:109:12:231
10930991414,0cyclictest27732-21kworker/1:108:51:011
10930991414,0cyclictest27732-21kworker/1:107:46:351
10930991414,0cyclictest27732-21kworker/1:107:45:011
10930991313,0cyclictest27732-21kworker/1:111:46:061
10930991313,0cyclictest27732-21kworker/1:111:39:001
10930991313,0cyclictest27732-21kworker/1:111:31:101
10930991313,0cyclictest27732-21kworker/1:111:18:341
10930991313,0cyclictest27732-21kworker/1:111:13:061
10930991313,0cyclictest27732-21kworker/1:111:02:061
10930991313,0cyclictest27732-21kworker/1:110:54:561
10930991313,0cyclictest27732-21kworker/1:110:44:581
10930991313,0cyclictest27732-21kworker/1:110:33:281
10930991313,0cyclictest27732-21kworker/1:110:01:091
10930991313,0cyclictest27732-21kworker/1:107:33:471
10930991313,0cyclictest27732-21kworker/1:107:16:091
10930991312,0cyclictest27732-21kworker/1:107:56:071
10930991212,0cyclictest27732-21kworker/1:112:20:481
10930991212,0cyclictest27732-21kworker/1:112:16:181
10930991212,0cyclictest27732-21kworker/1:112:14:081
10930991212,0cyclictest27732-21kworker/1:111:51:021
10930991212,0cyclictest27732-21kworker/1:111:29:081
10930991212,0cyclictest27732-21kworker/1:110:56:321
10930991212,0cyclictest27732-21kworker/1:110:56:321
10930991212,0cyclictest27732-21kworker/1:110:48:101
10930991212,0cyclictest27732-21kworker/1:110:25:541
10930991212,0cyclictest27732-21kworker/1:110:21:541
10930991212,0cyclictest27732-21kworker/1:110:09:371
10930991212,0cyclictest27732-21kworker/1:109:57:351
10930991212,0cyclictest27732-21kworker/1:109:49:031
10930991212,0cyclictest27732-21kworker/1:109:41:211
10930991212,0cyclictest27732-21kworker/1:109:37:271
10930991212,0cyclictest27732-21kworker/1:109:28:431
10930991212,0cyclictest27732-21kworker/1:109:01:191
10930991212,0cyclictest27732-21kworker/1:108:41:191
10930991212,0cyclictest27732-21kworker/1:108:36:031
10930991212,0cyclictest27732-21kworker/1:108:01:131
10930991212,0cyclictest27732-21kworker/1:107:26:131
10930991111,0cyclictest27732-21kworker/1:111:56:081
10930991111,0cyclictest27732-21kworker/1:109:16:071
10930991111,0cyclictest27732-21kworker/1:108:31:191
10930991111,0cyclictest27732-21kworker/1:108:21:211
10930991111,0cyclictest27732-21kworker/1:108:16:191
10930991111,0cyclictest27732-21kworker/1:108:11:071
10930991111,0cyclictest27732-21kworker/1:107:21:071
10930991111,0cyclictest27732-21kworker/1:107:13:211
10930991010,0cyclictest27732-21kworker/1:108:56:471
10930991010,0cyclictest27732-21kworker/1:107:51:091
10930991010,0cyclictest27732-21kworker/1:107:38:531
10930991010,0cyclictest27732-21kworker/1:107:05:531
109329983,0cyclictest29106-21munin-node11:06:242
109329983,0cyclictest28654-21iostat11:46:122
109329983,0cyclictest17748-21sed11:36:212
109329982,0cyclictest2058-21munin-node09:26:152
109329982,0cyclictest0-21swapper/211:51:552
109329982,0cyclictest0-21swapper/210:50:252
109329982,0cyclictest0-21swapper/209:54:462
109329974,0cyclictest2439-21chrt08:16:122
109329973,0cyclictest9033-21if_err_eth008:21:122
109329973,0cyclictest8893-21memory09:06:182
109329973,0cyclictest881-21fschecks_count07:51:092
109329973,0cyclictest8580-21munin-node12:31:202
109329973,0cyclictest7415-21fschecks_time10:31:112
109329973,0cyclictest6042-21perl07:26:092
109329973,0cyclictest5331-21memory10:06:152
109329973,0cyclictest4076-21df11:31:062
109329973,0cyclictest3708-21grep12:11:092
109329973,0cyclictest3316-21munin-node08:41:152
109329973,0cyclictest3285-21ssh11:30:272
109329973,0cyclictest32156-21if_eth010:26:122
109329973,0cyclictest31883-21hddtemp_smartct07:21:092
109329973,0cyclictest29854-21kthreadcore09:21:182
109329973,0cyclictest27848-21ssh12:06:112
109329973,0cyclictest27219-21aten2.4-expect12:26:062
109329973,0cyclictest26560-21sed07:46:082
109329973,0cyclictest26023-21diskstats09:41:092
109329973,0cyclictest25697-21ssh09:18:442
109329973,0cyclictest25015-21ntpq07:41:192
109329973,0cyclictest24653-21ssh09:38:402
109329973,0cyclictest24435-21ssh10:00:552
109329973,0cyclictest24065-21perl10:21:102
109329973,0cyclictest22118-21perl08:31:122
109329973,0cyclictest21231-21ssh11:00:552
109329973,0cyclictest20991-21hddtemp_smartct11:21:102
109329973,0cyclictest20377-21perl11:41:092
109329973,0cyclictest20190-21ssh10:58:122
109329973,0cyclictest20190-21ssh10:58:122
109329973,0cyclictest19991-21perl12:01:102
109329973,0cyclictest18875-21tail07:36:132
109329973,0cyclictest16531-21cut07:31:142
109329973,0cyclictest16447-21perl08:51:102
109329973,0cyclictest14881-21df10:36:062
109329973,0cyclictest14822-21grep12:16:142
109329973,0cyclictest14812-21cat08:25:492
109329973,0cyclictest13687-21cut07:06:132
109329973,0cyclictest12833-21entropy11:16:072
109329973,0cyclictest11362-21munin-node11:56:122
109329973,0cyclictest10584-21sh09:31:092
109329973,0cyclictest10267-21grep10:11:162
109329972,0cyclictest29220-21ssh10:43:492
109329972,0cyclictest15581-21munin-node10:16:212
109329972,0cyclictest0-21swapper/212:21:062
109329972,0cyclictest0-21swapper/211:11:302
109329972,0cyclictest0-21swapper/210:51:112
109329972,0cyclictest0-21swapper/209:56:322
109329972,0cyclictest0-21swapper/209:46:082
109329972,0cyclictest0-21swapper/209:11:062
109329972,0cyclictest0-21swapper/208:36:202
109329972,0cyclictest0-21swapper/208:01:072
109299974,0cyclictest0-21swapper/009:28:000
35160,0ktimersoftd/30-21swapper/311:56:493
339960,0migration/3323-21sh09:44:583
339960,0migration/326048-21sh12:23:303
339960,0migration/319241-21sh11:40:233
28405060,0irq/26-eth0-rx-20200-21hostname12:01:113
28405060,0irq/26-eth0-rx-17228-21ssh09:35:043
109329963,0cyclictest9457-21df08:46:072
109329963,0cyclictest7164-21sed07:55:492
109329963,0cyclictest29179-21stty09:01:062
109329963,0cyclictest27261-21aten_r1power_po08:11:072
109329963,0cyclictest24881-21forks07:16:082
109329963,0cyclictest22817-21diskstats08:56:082
109329963,0cyclictest20642-21aten2.4-expect08:06:052
109329963,0cyclictest18316-21fschecks_count07:11:112
109299965,0cyclictest11395-21df09:11:080
109299960,0cyclictest0-21swapper/008:56:100
109299960,0cyclictest0-21swapper/007:46:190
339950,0migration/39879-21taskset09:07:353
339950,0migration/38824-21latency_hist09:50:493
339950,0migration/38215-21unixbench_multi09:26:223
339950,0migration/37943-21munin-node08:41:203
339950,0migration/36987-21kthreadcore11:11:163
339950,0migration/36368-21if_eth107:26:113
339950,0migration/3454-21aten_r1power_en07:51:063
339950,0migration/33516-21kthreadcore08:16:133
339950,0migration/332188-21taskset10:05:563
339950,0migration/331360-21cpuspeed_turbos07:21:073
339950,0migration/329148-21kthreadcore08:11:133
339950,0migration/328804-21kthreadcore09:21:173
339950,0migration/327761-21kthreadcore07:46:133
339950,0migration/327443-21kthreadcore10:21:173
339950,0migration/325495-21kthreadcore08:56:133
339950,0migration/324713-21cpu10:01:093
339950,0migration/321046-21taskset08:28:033
339950,0migration/320697-21kthreadcore07:11:163
339950,0migration/319498-21rt-features08:01:183
339950,0migration/313336-21kthreadcore07:06:133
28405050,0irq/26-eth0-rx-8688-21open_inodes12:11:183
28405050,0irq/26-eth0-rx-7789-21iostat_ios10:31:143
28405050,0irq/26-eth0-rx-6547-21runrttasks11:08:493
28405050,0irq/26-eth0-rx-5533-21kthreadcore12:31:153
28405050,0irq/26-eth0-rx-3383-21snmpd09:15:593
28405050,0irq/26-eth0-rx-23365-21if_eth1.310:41:133
28405050,0irq/26-eth0-rx-15278-21latency_hist10:15:493
28405050,0irq/26-eth0-rx-13497-21awk10:56:063
28405050,0irq/26-eth0-rx-13497-21awk10:56:063
28405050,0irq/26-eth0-rx-10373-21seq11:53:563
28405050,0irq/26-eth0-rx-10292-21kthreadcore10:51:203
28405050,0irq/26-eth0-rx-0-21swapper/312:26:573
28405050,0irq/26-eth0-rx-0-21swapper/312:07:303
28405050,0irq/26-eth0-rx-0-21swapper/311:26:023
28405050,0irq/26-eth0-rx-0-21swapper/311:24:323
28405050,0irq/26-eth0-rx-0-21swapper/310:46:113
28405050,0irq/26-eth0-rx-0-21swapper/310:37:013
28405050,0irq/26-eth0-rx-0-21swapper/310:26:123
28405050,0irq/26-eth0-rx-0-21swapper/310:14:183
28405050,0irq/26-eth0-rx-0-21swapper/309:56:123
28405050,0irq/26-eth0-rx-0-21swapper/307:41:233
109299953,0cyclictest8889-21sadc08:45:490
109299952,0cyclictest8676-21sensors12:31:210
109299950,0cyclictest16170-21kthreadcore08:26:150
109299950,0cyclictest0-21swapper/012:26:140
109299950,0cyclictest0-21swapper/012:21:210
109299950,0cyclictest0-21swapper/012:01:220
109299950,0cyclictest0-21swapper/011:56:350
109299950,0cyclictest0-21swapper/011:46:140
109299950,0cyclictest0-21swapper/011:36:210
109299950,0cyclictest0-21swapper/011:26:190
109299950,0cyclictest0-21swapper/011:16:120
109299950,0cyclictest0-21swapper/011:11:130
109299950,0cyclictest0-21swapper/011:06:110
109299950,0cyclictest0-21swapper/011:03:380
109299950,0cyclictest0-21swapper/010:59:070
109299950,0cyclictest0-21swapper/010:59:070
109299950,0cyclictest0-21swapper/010:21:200
109299950,0cyclictest0-21swapper/010:16:210
109299950,0cyclictest0-21swapper/010:11:160
109299950,0cyclictest0-21swapper/009:41:150
109299950,0cyclictest0-21swapper/009:21:170
109299950,0cyclictest0-21swapper/009:06:160
109299950,0cyclictest0-21swapper/008:32:290
109299950,0cyclictest0-21swapper/008:16:150
109299950,0cyclictest0-21swapper/008:11:140
109299950,0cyclictest0-21swapper/008:00:490
109299950,0cyclictest0-21swapper/007:31:130
109299950,0cyclictest0-21swapper/007:06:130
339940,0migration/34708-21kthreadcore09:46:133
339940,0migration/322270-21kthreadcore09:36:133
339940,0migration/321817-21kthreadcore11:41:123
339940,0migration/320470-21latency_hist08:05:493
28405040,0irq/26-eth0-rx-0-21swapper/312:17:253
28405040,0irq/26-eth0-rx-0-21swapper/311:45:573
28405040,0irq/26-eth0-rx-0-21swapper/311:31:053
28405040,0irq/26-eth0-rx-0-21swapper/311:18:053
28405040,0irq/26-eth0-rx-0-21swapper/311:02:503
28405040,0irq/26-eth0-rx-0-21swapper/309:11:103
28405040,0irq/26-eth0-rx-0-21swapper/309:01:173
28405040,0irq/26-eth0-rx-0-21swapper/308:51:143
28405040,0irq/26-eth0-rx-0-21swapper/308:46:063
28405040,0irq/26-eth0-rx-0-21swapper/308:36:083
28405040,0irq/26-eth0-rx-0-21swapper/307:31:073
109299943,0cyclictest28281-21awk12:06:130
109299940,0cyclictest9082sleep009:45:580
109299940,0cyclictest31122-21munin-node07:21:080
109299940,0cyclictest24467-21munin-node07:16:070
109299940,0cyclictest22307-21sh10:40:510
109299940,0cyclictest19573-21aten_r1power_cu07:41:090
109299940,0cyclictest11038-21rm12:15:550
109299940,0cyclictest0-21swapper/012:11:070
109299940,0cyclictest0-21swapper/011:51:090
109299940,0cyclictest0-21swapper/011:41:050
109299940,0cyclictest0-21swapper/011:31:050
109299940,0cyclictest0-21swapper/011:21:080
109299940,0cyclictest0-21swapper/010:51:070
109299940,0cyclictest0-21swapper/010:45:490
109299940,0cyclictest0-21swapper/010:36:000
109299940,0cyclictest0-21swapper/010:31:110
109299940,0cyclictest0-21swapper/010:25:570
109299940,0cyclictest0-21swapper/010:05:490
109299940,0cyclictest0-21swapper/010:01:080
109299940,0cyclictest0-21swapper/009:55:490
109299940,0cyclictest0-21swapper/009:51:090
109299940,0cyclictest0-21swapper/009:36:080
109299940,0cyclictest0-21swapper/009:31:090
109299940,0cyclictest0-21swapper/009:16:060
109299940,0cyclictest0-21swapper/009:00:490
109299940,0cyclictest0-21swapper/008:51:110
109299940,0cyclictest0-21swapper/008:46:100
109299940,0cyclictest0-21swapper/008:35:520
109299940,0cyclictest0-21swapper/008:21:090
109299940,0cyclictest0-21swapper/008:06:050
109299940,0cyclictest0-21swapper/008:01:070
109299940,0cyclictest0-21swapper/007:50:490
109299940,0cyclictest0-21swapper/007:26:080
109299940,0cyclictest0-21swapper/007:11:080
28405030,0irq/26-eth0-rx-0-21swapper/308:33:313
28405030,0irq/26-eth0-rx-0-21swapper/307:56:073
109299930,0cyclictest0-21swapper/007:35:490
329920,0watchdog/324865-21entropy07:16:073
28405020,0irq/26-eth0-rx-0-21swapper/308:22:193
28405020,0irq/26-eth0-rx-0-21swapper/307:38:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional