You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-20 - 00:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Apr 19, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2509023627,0sleep30-21swapper/306:59:273
2518423521,0sleep00-21swapper/007:00:320
2518123526,0sleep20-21swapper/207:00:302
2487423526,0sleep10-21swapper/106:56:571
28415070,0irq/27-eth0-tx-0-21swapper/212:01:202
28415070,0irq/27-eth0-tx-0-21swapper/211:07:502
28415070,0irq/27-eth0-tx-0-21swapper/209:22:252
339960,0migration/34229-21sh09:48:173
339960,0migration/319943-21sh12:09:363
339960,0migration/319478-21munin-node07:41:193
28415060,0irq/27-eth0-tx-28037-21ssh12:15:062
28415060,0irq/27-eth0-tx-27272-21ssh11:50:452
28415060,0irq/27-eth0-tx-23000-21telnet08:36:100
28415060,0irq/27-eth0-tx-0-21swapper/212:23:102
28415060,0irq/27-eth0-tx-0-21swapper/211:58:032
28415060,0irq/27-eth0-tx-0-21swapper/211:51:212
28415060,0irq/27-eth0-tx-0-21swapper/211:41:252
28415060,0irq/27-eth0-tx-0-21swapper/211:34:222
28415060,0irq/27-eth0-tx-0-21swapper/211:27:312
28415060,0irq/27-eth0-tx-0-21swapper/211:23:002
28415060,0irq/27-eth0-tx-0-21swapper/211:17:392
28415060,0irq/27-eth0-tx-0-21swapper/211:12:122
28415060,0irq/27-eth0-tx-0-21swapper/210:56:152
28415060,0irq/27-eth0-tx-0-21swapper/210:52:372
28415060,0irq/27-eth0-tx-0-21swapper/210:46:182
28415060,0irq/27-eth0-tx-0-21swapper/210:42:502
28415060,0irq/27-eth0-tx-0-21swapper/210:37:272
28415060,0irq/27-eth0-tx-0-21swapper/209:56:092
28415060,0irq/27-eth0-tx-0-21swapper/209:35:182
28415060,0irq/27-eth0-tx-0-21swapper/209:08:552
28405060,0irq/26-eth0-rx-25232-21kthreadcore09:21:183
28405060,0irq/26-eth0-rx-23037-21cat11:26:173
28405060,0irq/26-eth0-rx-14175-40yum-updatesd-he10:21:043
252939966,0cyclictest27647-21sensors09:41:211
252939966,0cyclictest23624-21diskmemload09:13:481
249960,0watchdog/227445-21ssh10:01:412
249960,0watchdog/20-21swapper/209:52:532
249960,0watchdog/20-21swapper/209:38:092
249960,0watchdog/20-21swapper/209:26:372
149960,0migration/026699-21sh11:26:560
149960,0migration/023920-21kthreadcore07:46:200
149960,0migration/014433-21kthreadcore11:41:190
8225050,0irq/24-0000:00:2515-21cat08:20:582
339950,0migration/34092-21kthreadcore07:56:173
339950,0migration/332354-21kthreadcore12:16:143
339950,0migration/331219-21kthreadcore07:26:173
339950,0migration/329851-21kthreadcore11:31:143
339950,0migration/327000-21kthreadcore07:01:153
339950,0migration/326900-21taskset07:47:233
339950,0migration/32587-21kthreadcore07:06:183
339950,0migration/325848-21taskset07:21:333
339950,0migration/322247-21aten_r1power_po10:26:103
339950,0migration/32124-21kthreadcore09:06:213
339950,0migration/319528-21kthreadcore08:31:163
339950,0migration/319115-21sh11:02:233
339950,0migration/318855-21kthreadcore08:06:183
339950,0migration/318720-21kthreadcore09:36:173
339950,0migration/315671-21kthreadcore08:51:193
339950,0migration/315600-21taskset08:26:453
339950,0migration/31552-21latency_hist07:55:583
339950,0migration/315111-21kthreadcore11:41:193
339950,0migration/314367-21munin-node10:41:153
339950,0migration/31189-21kthreadcore10:11:123
339950,0migration/311667-21kthreadcore07:36:163
28415050,0irq/27-eth0-tx-13313-21ssh09:11:272
28415050,0irq/27-eth0-tx-0-21swapper/212:16:542
28415050,0irq/27-eth0-tx-0-21swapper/212:06:332
28415050,0irq/27-eth0-tx-0-21swapper/210:31:422
28415050,0irq/27-eth0-tx-0-21swapper/210:21:132
28415050,0irq/27-eth0-tx-0-21swapper/210:06:282
28415050,0irq/27-eth0-tx-0-21swapper/209:47:402
28415050,0irq/27-eth0-tx-0-21swapper/209:16:012
28415050,0irq/27-eth0-tx-0-21swapper/008:51:220
28415050,0irq/27-eth0-tx-0-21swapper/008:47:250
28415050,0irq/27-eth0-tx-0-21swapper/008:27:310
28415050,0irq/27-eth0-tx-0-21swapper/008:01:120
28415050,0irq/27-eth0-tx-0-21swapper/007:51:160
28405050,0irq/26-eth0-rx-9501-21kthreadcore12:01:203
28405050,0irq/26-eth0-rx-5932-21forks10:36:093
28405050,0irq/26-eth0-rx-29186-21ssh10:06:273
28405050,0irq/26-eth0-rx-21882-21ssh09:41:083
28405050,0irq/26-eth0-rx-21114-21grep11:46:193
28405050,0irq/26-eth0-rx-21011-21diskstats11:06:093
28405050,0irq/26-eth0-rx-13023-21fschecks_time11:21:133
28405050,0irq/26-eth0-rx-0-21swapper/312:29:163
28405050,0irq/26-eth0-rx-0-21swapper/312:13:163
28405050,0irq/26-eth0-rx-0-21swapper/311:52:303
28405050,0irq/26-eth0-rx-0-21swapper/311:00:043
28405050,0irq/26-eth0-rx-0-21swapper/310:31:373
28405050,0irq/26-eth0-rx-0-21swapper/310:01:463
28405050,0irq/26-eth0-rx-0-21swapper/309:54:463
28405050,0irq/26-eth0-rx-0-21swapper/309:32:343
28405050,0irq/26-eth0-rx-0-21swapper/309:13:123
28405050,0irq/26-eth0-rx-0-21swapper/309:00:593
252949950,0cyclictest0-21swapper/210:11:552
252949950,0cyclictest0-21swapper/208:02:102
252939955,0cyclictest0-21swapper/111:59:111
252939955,0cyclictest0-21swapper/110:57:421
252939950,0cyclictest0-21swapper/111:29:051
252939950,0cyclictest0-21swapper/110:22:331
252939950,0cyclictest0-21swapper/110:09:281
252919950,0cyclictest0-21swapper/009:40:040
252919950,0cyclictest0-21swapper/009:00:590
259950,0migration/25716-21kthreadcore07:31:152
259950,0migration/226275-21open_files07:46:222
259950,0migration/225246-21munin-node07:21:202
259950,0migration/214504-21kthreadcore08:26:162
249950,0watchdog/215135-21aten_r1power_cu08:06:092
249950,0watchdog/20-21swapper/212:26:132
249950,0watchdog/20-21swapper/211:04:212
249950,0watchdog/20-21swapper/210:26:292
249950,0watchdog/20-21swapper/210:18:092
249950,0watchdog/20-21swapper/209:44:052
249950,0watchdog/20-21swapper/208:58:092
249950,0watchdog/20-21swapper/208:51:052
249950,0watchdog/20-21swapper/208:32:332
249950,0watchdog/20-21swapper/208:21:132
249950,0watchdog/20-21swapper/207:43:052
249950,0watchdog/20-21swapper/207:27:172
249950,0watchdog/20-21swapper/207:17:092
19150,0ktimersoftd/10-21swapper/112:05:461
179950,0migration/18544-21kthreadcore10:16:181
179950,0migration/16878-21kthreadcore07:56:181
179950,0migration/128891-21unixbench-2d09:21:271
179950,0migration/127789-21pmu-power10:26:221
179950,0migration/125554-21kthreadcore09:01:171
179950,0migration/123680-21kthreadcore11:46:201
179950,0migration/113187-21munin-node07:36:231
179950,0migration/11034-21sh11:51:231
169950,0watchdog/130350-21irqstats10:51:171
169950,0watchdog/124900-21grep08:36:171
169950,0watchdog/117094-21grep09:56:171
149950,0migration/09654-21kthreadcore10:36:150
149950,0migration/0763-21kthreadcore10:11:120
149950,0migration/04774-21kthreadcore08:21:250
149950,0migration/03296-21kthreadcore07:56:170
149950,0migration/030088-21kthreadcore11:31:140
149950,0migration/027224-21timerwakeupswit10:46:220
149950,0migration/026680-21missed_timers10:01:200
149950,0migration/02594-21kthreadcore09:26:150
149950,0migration/02551-21needreboot08:41:190
149950,0migration/023438-21kthreadcore12:11:190
149950,0migration/023313-21kthreadcore07:21:180
149950,0migration/020187-21kthreadcore08:06:190
149950,0migration/019873-21timerandwakeup07:41:250
149950,0migration/017418-21kthreadcore09:16:160
149950,0migration/017055-21kthreadcore12:06:180
149950,0migration/016256-21kthreadcore11:21:200
149950,0migration/016111-21kthreadcore12:26:200
149950,0migration/016017-21kthreadcore09:56:170
149950,0migration/013321-21taskset09:34:250
149950,0migration/011127-21timerandwakeup12:01:270
149950,0migration/010493-21timerandwakeup10:56:210
339940,0migration/39508-21kthreadcore10:16:193
339940,0migration/35464-21kthreadcore08:46:173
339940,0migration/324778-21kthreadcore10:46:163
339940,0migration/32330-21taskset08:19:423
339940,0migration/31990-21kthreadcore08:41:183
339940,0migration/31819-21irqcore07:31:133
339940,0migration/315556-21kthreadcore07:16:153
28415040,0irq/27-eth0-tx-13349-21telnet07:16:090
28415040,0irq/27-eth0-tx-0-21swapper/211:36:202
28415040,0irq/27-eth0-tx-0-21swapper/008:16:130
28405040,0irq/26-eth0-rx-0-21swapper/312:21:073
28405040,0irq/26-eth0-rx-0-21swapper/311:56:313
28405040,0irq/26-eth0-rx-0-21swapper/311:36:443
28405040,0irq/26-eth0-rx-0-21swapper/311:16:043
28405040,0irq/26-eth0-rx-0-21swapper/311:13:393
28405040,0irq/26-eth0-rx-0-21swapper/310:53:403
28405040,0irq/26-eth0-rx-0-21swapper/309:56:343
28405040,0irq/26-eth0-rx-0-21swapper/309:28:083
28405040,0irq/26-eth0-rx-0-21swapper/309:18:573
252949943,0cyclictest3425-21lldpd07:56:042
252939944,0cyclictest0-21swapper/107:01:131
252939943,0cyclictest7307-21kworker/1:208:41:221
252939943,0cyclictest4826-21awk08:46:151
252939943,0cyclictest3425-21lldpd11:15:341
252939943,0cyclictest3425-21lldpd09:20:511
252939943,0cyclictest3425-21lldpd08:21:261
252939943,0cyclictest30886-21awk10:31:151
252939943,0cyclictest28368-21awk07:51:201
252939943,0cyclictest23613-21kworker/1:112:29:261
252939943,0cyclictest23613-21kworker/1:111:36:381
252939943,0cyclictest23613-21kworker/1:110:37:401
252939943,0cyclictest23613-21kworker/1:110:04:281
252939943,0cyclictest23613-21kworker/1:109:53:001
252939943,0cyclictest23613-21kworker/1:109:46:001
252939943,0cyclictest23613-21kworker/1:109:36:081
252939943,0cyclictest23613-21kworker/1:109:34:361
252939943,0cyclictest12726-21awk11:41:161
252919943,0cyclictest6445-21snmpd10:33:500
252919943,0cyclictest6445-21snmpd10:29:500
252919943,0cyclictest3425-21lldpd11:16:350
252919943,0cyclictest3425-21lldpd10:16:170
252919943,0cyclictest3425-21lldpd08:59:040
252919943,0cyclictest3110-21irqbalance11:56:530
252919943,0cyclictest3110-21irqbalance11:51:130
252919943,0cyclictest3110-21irqbalance11:46:030
252919943,0cyclictest3110-21irqbalance09:12:430
252919943,0cyclictest3110-21irqbalance08:31:230
252919943,0cyclictest3110-21irqbalance08:12:030
252919943,0cyclictest3110-21irqbalance07:31:030
252919943,0cyclictest3110-21irqbalance07:26:030
252919943,0cyclictest3110-21irqbalance07:06:030
252919943,0cyclictest3110-21irqbalance07:02:030
252919943,0cyclictest30056-21awk10:51:160
252919942,0cyclictest0-21swapper/007:11:110
252919942,0cyclictest0-21swapper/007:11:110
252919940,0cyclictest7458-21munin-node07:36:130
252919940,0cyclictest0-21swapper/011:07:160
259940,0migration/23864-21munin-node08:46:092
259940,0migration/224346-21kthreadcore08:36:172
259940,0migration/222056-21cpuspeed_turbos08:11:102
249940,0watchdog/27836-21kthreadcore07:11:172
249940,0watchdog/27836-21kthreadcore07:11:172
249940,0watchdog/20-21swapper/209:01:052
249940,0watchdog/20-21swapper/208:45:252
249940,0watchdog/20-21swapper/207:51:172
249940,0watchdog/20-21swapper/207:36:012
249940,0watchdog/20-21swapper/207:06:212
249940,0watchdog/20-21swapper/207:01:212
179940,0migration/19239-21kthreadcore12:21:171
179940,0migration/17342-21idleruntime07:11:141
179940,0migration/17342-21idleruntime07:11:141
179940,0migration/16217-21ntp_kernel_err07:31:171
179940,0migration/13555-21sshd11:35:491
179940,0migration/132460-21cpuspeed_turbos07:06:121
179940,0migration/131408-21fschecks_time10:11:081
179940,0migration/130318-21kthreadcore12:16:141
179940,0migration/126238-21sh11:06:191
179940,0migration/122685-21idleruntime-cro08:35:581
179940,0migration/122410-21kthreadcore12:11:181
179940,0migration/119473-21taskset11:24:191
179940,0migration/118192-21kthreadcore07:16:161
179940,0migration/11179-21kthreadcore09:06:201
179940,0migration/111186-21kthreadcore08:26:151
179940,0migration/111096-21kthreadcore08:01:171
169940,0watchdog/14740-21ssh09:27:331
169940,0watchdog/14307-21ssh11:16:011
169940,0watchdog/126879-21ps07:47:091
169940,0watchdog/119102-21ssh11:02:211
169940,0watchdog/119012-21ssh10:41:211
169940,0watchdog/113809-21fschecks_time12:06:131
169940,0watchdog/10-21swapper/110:46:171
169940,0watchdog/10-21swapper/108:16:171
169940,0watchdog/10-21swapper/107:26:171
149940,0migration/05433-21irqrtprio12:21:160
149940,0migration/032168-21kthreadcore12:16:140
149940,0migration/027622-21kthreadcore09:21:190
149940,0migration/018804-21kthreadcore10:21:170
28405030,0irq/26-eth0-rx-0-21swapper/308:36:103
28405030,0irq/26-eth0-rx-0-21swapper/308:21:093
28405030,0irq/26-eth0-rx-0-21swapper/308:01:093
28405030,0irq/26-eth0-rx-0-21swapper/307:11:113
28405030,0irq/26-eth0-rx-0-21swapper/307:11:113
252939933,0cyclictest7307-21kworker/1:208:56:081
252939933,0cyclictest7307-21kworker/1:208:51:101
252939933,0cyclictest7307-21kworker/1:208:11:101
252939933,0cyclictest7307-21kworker/1:207:41:001
252919932,0cyclictest3110-21irqbalance10:09:430
252919932,0cyclictest30689-21awk09:46:130
252919932,0cyclictest29611-21awk11:11:160
252919932,0cyclictest2833-21kthreadcore09:06:210
252919932,0cyclictest18103-21grep10:41:190
252919932,0cyclictest13757-21cut11:01:140
252919932,0cyclictest11234-21sensors09:51:250
252919931,0cyclictest38802chrt11:36:050
169930,0watchdog/10-21swapper/108:06:091
169930,0watchdog/10-21swapper/107:21:051
149930,0migration/028199-21sh09:41:580
28405020,0irq/26-eth0-rx-0-21swapper/308:56:163
28405020,0irq/26-eth0-rx-0-21swapper/308:12:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional