You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-22 - 06:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot4.osadl.org (updated Fri Sep 22, 2023 00:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1493799674,37cyclictest0-21swapper/120:28:341
14951995617,32cyclictest18054-21ntpq22:10:343
1494599564,46cyclictest0-21swapper/223:05:462
1493799564,45cyclictest0-21swapper/120:58:391
1494599553,46cyclictest0-21swapper/200:01:152
14945995512,38cyclictest905-21ssh22:33:382
1458825510,38sleep10-21swapper/118:48:461
1493799544,44cyclictest0-21swapper/122:45:021
1493799544,44cyclictest0-21swapper/121:36:281
1493799542,46cyclictest30915-21ssh22:30:171
14951995324,23cyclictest22754-21tune2fs20:35:253
14951995316,31cyclictest6431-21iostat_ios21:05:283
14945995311,36cyclictest20183-21ssh21:25:362
14945995310,38cyclictest1807-21grep21:45:392
1493799534,43cyclictest0-21swapper/122:01:481
1493799533,9cyclictest0-21swapper/121:41:431
14937995313,35cyclictest2921-21ksmtuned21:48:341
1493399534,44cyclictest0-21swapper/022:15:380
14951995224,23cyclictest15682-21awk20:15:313
1495199522,44cyclictest18813-21wc20:25:223
14951995219,28cyclictest17750-21smart_sda20:20:413
14951995216,30cyclictest2502-21missed_timers19:40:353
14951995214,32cyclictest24520-21cut20:40:233
1494599524,42cyclictest0-21swapper/221:03:462
1494599524,42cyclictest0-21swapper/221:03:462
14945995212,35cyclictest1406-21ssh00:07:562
1493799524,43cyclictest0-21swapper/100:16:531
1493799524,42cyclictest0-21swapper/123:17:161
14933995243,5cyclictest3-21ksoftirqd/019:40:050
1493399524,42cyclictest0-21swapper/021:35:290
1495199519,35cyclictest21387-21netstat22:15:303
1495199513,41cyclictest7233-21grep00:15:403
14951995120,26cyclictest7891-21hwlatdetect19:55:263
14951995120,26cyclictest7891-21hwlatdetect19:55:263
14951995120,26cyclictest4148-21fschecks_count23:25:223
1495199512,42cyclictest28898-21tr00:00:313
14951995118,28cyclictest12100-21grep22:50:163
14951995116,29cyclictest29833-21sshd23:15:303
14951995116,29cyclictest12439-21grep20:05:433
14951995115,31cyclictest6824-21unixbench_singl19:50:423
14951995115,31cyclictest13583-21cut21:15:383
14951995114,32cyclictest31530-21timerandwakeup19:30:443
14951995114,31cyclictest27273-21kernelversion19:20:313
14951995112,33cyclictest16800-21sed22:55:393
14951995111,34cyclictest644-21runrttasks23:50:383
14951995111,34cyclictest14163-21fschecks_count22:05:243
14951995110,35cyclictest5660-21runrttasks21:51:553
1494599519,36cyclictest22975-21ssh23:51:182
1494599518,10cyclictest27396-21unixbench_singl21:35:422
1494599515,39cyclictest0-21swapper/222:56:562
1494599515,12cyclictest0-21swapper/223:10:302
1494599513,43cyclictest0-21swapper/222:43:372
1494599511,44cyclictest8333-21ssh21:08:292
1494599510,45cyclictest3125-21ssh00:10:332
1494599510,45cyclictest3125-21ssh00:10:332
1493799516,39cyclictest0-21swapper/121:50:361
1493799515,40cyclictest0-21swapper/122:24:461
1493799515,39cyclictest0-21swapper/123:55:401
1493799515,39cyclictest0-21swapper/120:50:041
1493799513,43cyclictest0-21swapper/122:53:521
1493799513,43cyclictest0-21swapper/121:10:121
1493799513,43cyclictest0-21swapper/121:00:391
1493799513,43cyclictest0-21swapper/121:00:391
1493799513,43cyclictest0-21swapper/120:04:421
1493799513,42cyclictest14505-21ssh21:17:011
1493799512,18cyclictest566-21gdbus20:55:031
1495199508,34cyclictest25769-21rt-features19:15:393
1495199503,9cyclictest32524-21sendmail_mailqu20:55:393
1495199503,41cyclictest3912-21diskstats19:45:253
14951995019,26cyclictest9274-21grep21:10:203
14951995016,29cyclictest29560-21switchtime19:25:423
14951995016,28cyclictest31051-21fschecks_time22:30:203
14951995015,29cyclictest9771-21idleruntime20:00:253
14951995015,29cyclictest30041-21latency21:40:283
14951995015,28cyclictest644-21runrttasks00:08:013
14951995012,33cyclictest8627-21timerandwakeup23:30:393
14951995012,32cyclictest11932-21ntpq23:35:333
14951995011,32cyclictest1262-21ntpq23:20:353
14951995010,35cyclictest21669-21/usr/sbin/munin19:05:333
14951995010,34cyclictest644-21runrttasks23:03:463
1494599508,36cyclictest13666-21ssh22:51:102
1494599506,37cyclictest0-21swapper/223:45:442
1494599505,39cyclictest281ktimersoftd/219:05:272
1494599505,39cyclictest0-21swapper/221:40:132
1494599505,39cyclictest0-21swapper/221:24:332
1494599503,10cyclictest0-21swapper/222:12:482
14945995012,33cyclictest9543-21ssh21:58:182
14945995012,33cyclictest9543-21ssh21:58:182
1493799509,36cyclictest0-21swapper/122:27:471
1493799509,35cyclictest0-21swapper/121:20:401
1493799508,37cyclictest0-21swapper/122:38:561
1493799504,9cyclictest0-21swapper/123:29:061
1493799504,41cyclictest0-21swapper/123:41:061
1493799504,40cyclictest0-21swapper/123:14:481
1493799504,40cyclictest0-21swapper/121:29:591
1493799504,14cyclictest0-21swapper/119:15:331
14933995012,33cyclictest15768-21timerandwakeup23:40:420
1495199494,8cyclictest0-21swapper/319:39:193
1495199494,6cyclictest0-21swapper/320:55:033
1495199494,16cyclictest0-21swapper/321:55:283
1495199494,16cyclictest0-21swapper/321:55:283
1495199493,7cyclictest20093-21/usr/sbin/munin19:00:393
1495199493,38cyclictest16109-21aten_rbpower_vo21:20:213
14951994918,26cyclictest27430-21grep22:25:163
14951994918,25cyclictest20138-21sshd21:25:323
14951994917,27cyclictest23450-21idleruntime19:10:273
14951994916,28cyclictest644-21runrttasks23:42:493
14951994915,29cyclictest21566-21timerandwakeup20:30:383
14951994915,28cyclictest13652-21latency20:10:303
14951994913,30cyclictest26369-21entropy20:45:233
14951994911,33cyclictest3713-21timerwakeupswit21:00:383
14951994911,33cyclictest3713-21timerwakeupswit21:00:383
14951994911,33cyclictest25851-21timerandwakeup23:55:383
1494599498,35cyclictest0-21swapper/222:38:282
1494599496,37cyclictest0-21swapper/218:55:062
1494599495,16cyclictest0-21swapper/221:19:382
1494599494,40cyclictest0-21swapper/223:38:462
1494599494,39cyclictest0-21swapper/223:01:322
1494599494,39cyclictest0-21swapper/221:51:332
1494599494,39cyclictest0-21swapper/221:12:222
1494599494,39cyclictest0-21swapper/220:52:462
1494599494,38cyclictest0-21swapper/223:40:062
1494599493,41cyclictest0-21swapper/222:00:082
1494599493,41cyclictest0-21swapper/221:30:392
14945994913,31cyclictest7058-21cat00:15:382
14945994910,6cyclictest0-21swapper/223:16:452
14945994910,34cyclictest9492-21open_inodes22:45:322
14945994910,34cyclictest9492-21open_inodes22:45:322
1493799497,36cyclictest0-21swapper/123:45:091
1493799494,40cyclictest0-21swapper/123:23:371
1493799494,40cyclictest0-21swapper/119:20:231
1493799494,39cyclictest0-21swapper/123:36:131
1493799494,39cyclictest0-21swapper/123:33:111
1493799493,7cyclictest0-21swapper/100:06:351
1493799493,41cyclictest0-21swapper/123:50:491
1493799493,41cyclictest0-21swapper/121:55:281
1493799493,41cyclictest0-21swapper/121:55:281
1493799493,40cyclictest0-21swapper/122:57:511
1493799493,10cyclictest0-21swapper/122:47:471
1493799493,10cyclictest0-21swapper/122:47:471
14937994910,34cyclictest0-21swapper/122:18:501
14951994815,27cyclictest1471-21switchtime21:45:363
14951994814,28cyclictest3067-21if_eth200:10:303
14951994814,28cyclictest3067-21if_eth200:10:303
14951994813,30cyclictest16505-21switchtime18:50:383
14951994813,29cyclictest6264-21timerandwakeup22:40:383
1494599488,34cyclictest0-21swapper/223:31:582
1494599488,34cyclictest0-21swapper/223:25:042
1494599488,34cyclictest0-21swapper/223:21:292
1494599486,36cyclictest17752-21perl18:55:282
1494599484,38cyclictest28219-21open_files22:25:312
1494599484,38cyclictest0-21swapper/223:58:332
1494599484,38cyclictest0-21swapper/220:59:142
1494599482,40cyclictest19290-21/usr/sbin/munin19:00:392
14945994812,31cyclictest14280-21tune2fs22:05:242
14945994812,30cyclictest24636-21ssh22:20:292
1493799488,34cyclictest0-21swapper/123:03:461
1493799485,11cyclictest0-21swapper/120:35:361
1493799484,38cyclictest0-21swapper/121:09:451
1493799484,38cyclictest0-21swapper/119:10:371
1493799483,39cyclictest0-21swapper/122:12:281
14937994810,6cyclictest0-21swapper/100:01:491
1493399484,39cyclictest0-21swapper/023:45:290
1495199474,14cyclictest0-21swapper/322:45:383
1495199474,14cyclictest0-21swapper/322:45:383
1495199473,7cyclictest0-21swapper/323:10:173
1495199472,8cyclictest18587-21/usr/sbin/munin18:55:453
1495199472,40cyclictest10869-21cut22:00:273
14951994715,28cyclictest644-21runrttasks21:30:343
14951994715,28cyclictest22665-21fschecks_count23:05:223
14951994714,27cyclictest26616-21iostat21:35:283
14951994714,27cyclictest18226-21fschecks_count23:45:233
14951994713,29cyclictest24144-21cpu22:20:203
1494599478,33cyclictest21165-21iostat_ios22:15:262
1493799474,38cyclictest0-21swapper/122:05:081
1493799474,37cyclictest0-21swapper/123:05:231
1493799474,37cyclictest0-21swapper/100:11:231
1493799474,37cyclictest0-21swapper/100:11:231
1493399474,38cyclictest0-21swapper/023:03:490
1493399472,39cyclictest30171-21missed_timers21:40:300
14951994610,32cyclictest1877-21cpu22:35:193
1493799464,37cyclictest0-21swapper/121:34:191
1493799464,37cyclictest0-21swapper/120:40:251
1493799464,37cyclictest0-21swapper/120:08:231
1493799464,37cyclictest0-21swapper/119:40:451
1493799464,37cyclictest0-21swapper/119:30:441
1493799464,11cyclictest0-21swapper/120:20:241
1493799463,36cyclictest0-21swapper/120:30:261
1493399464,37cyclictest0-21swapper/022:40:330
1493399464,37cyclictest0-21swapper/021:52:280
1493399464,36cyclictest0-21swapper/021:25:520
1493799454,36cyclictest0-21swapper/119:38:291
1493799454,35cyclictest0-21swapper/119:25:371
1493799453,37cyclictest0-21swapper/120:15:251
1493799453,37cyclictest0-21swapper/119:51:131
1493399458,7cyclictest0-21swapper/020:58:220
1493399455,33cyclictest0-21swapper/023:15:400
1493399454,35cyclictest0-21swapper/023:30:370
1493399453,36cyclictest0-21swapper/022:35:300
1493399452,36cyclictest23369-21/usr/sbin/munin23:05:340
1493799444,8cyclictest0-21swapper/119:45:241
1493799444,35cyclictest0-21swapper/119:01:401
1493799444,17cyclictest0-21swapper/119:05:331
1493799443,36cyclictest0-21swapper/119:55:211
1493799443,36cyclictest0-21swapper/119:55:211
1493799443,36cyclictest0-21swapper/118:53:411
1493799443,35cyclictest0-21swapper/120:10:311
1493399445,7cyclictest0-21swapper/022:53:290
1493399445,33cyclictest13753-21timerandwakeup21:15:410
1493399445,33cyclictest0-21swapper/023:11:390
1493399444,35cyclictest0-21swapper/023:57:030
1493399444,35cyclictest0-21swapper/022:45:530
1493399444,35cyclictest0-21swapper/022:45:530
1493399444,34cyclictest0-21swapper/021:59:380
1493399444,34cyclictest0-21swapper/021:59:380
1493399443,35cyclictest0-21swapper/023:25:370
1493799434,35cyclictest0-21swapper/118:57:561
1493399435,32cyclictest0-21swapper/022:55:480
1493399434,34cyclictest0-21swapper/023:24:530
1493399434,32cyclictest0-21swapper/000:00:480
1493399434,14cyclictest0-21swapper/021:02:180
1493399434,14cyclictest0-21swapper/021:02:180
1493399423,33cyclictest29014-21ssh20:50:350
1493399402,33cyclictest5292-21ssh00:14:090
1493399402,33cyclictest5292-21ssh00:14:090
1455623914,17sleep20-21swapper/218:48:272
137372398,23sleep00-21swapper/018:45:310
282312384,8sleep20-21swapper/219:23:032
183532383,6sleep00-21swapper/020:23:380
170932383,4sleep00-21swapper/018:53:560
206982373,6sleep00-21swapper/019:02:210
14933993728,5cyclictest3-21ksoftirqd/020:50:030
319942363,6sleep00-21swapper/019:33:230
1493399365,26cyclictest0-21swapper/023:38:070
1493399364,28cyclictest0-21swapper/021:48:110
14933993624,7cyclictest19904-21sed20:25:420
88202353,8sleep20-21swapper/219:56:312
88202353,8sleep20-21swapper/219:56:312
71272353,7sleep00-21swapper/019:53:360
58752353,6sleep00-21swapper/021:05:190
1493399354,26cyclictest0-21swapper/022:00:060
1493399354,26cyclictest0-21swapper/019:15:430
1435223510,19sleep30-21swapper/318:45:583
14933993423,7cyclictest10355-21sort20:00:380
128332343,6sleep00-21swapper/020:09:490
126592343,8sleep20-21swapper/220:07:422
256042333,6sleep00-21swapper/020:41:000
1494599334,23cyclictest0-21swapper/220:24:062
1494599334,23cyclictest0-21swapper/219:50:272
14945993310,17cyclictest16191-21unixbench_singl20:15:402
1493399334,25cyclictest0-21swapper/000:05:370
1493399334,24cyclictest0-21swapper/022:23:360
14933993322,7cyclictest18606-21sed22:10:420
14933993320,8cyclictest22821-21cut21:30:190
14933993316,13cyclictest14156-21timerwakeupswit20:10:400
224282324,6sleep00-21swapper/020:35:170
1494599328,18cyclictest22212-21munin-run20:35:032
14945993220,7cyclictest24893-21/usr/sbin/munin19:15:242
1493399324,24cyclictest0-21swapper/022:26:130
1493399324,24cyclictest0-21swapper/019:26:430
1493399324,24cyclictest0-21swapper/019:22:180
1493399324,24cyclictest0-21swapper/000:16:000
1493399324,23cyclictest0-21swapper/023:51:370
1493399322,25cyclictest13955-21cut22:05:200
1494599314,22cyclictest0-21swapper/220:35:062
1494599314,22cyclictest0-21swapper/219:35:372
1494599314,21cyclictest0-21swapper/219:40:372
1494599314,21cyclictest0-21swapper/219:12:062
1493399314,23cyclictest0-21swapper/022:30:360
14933993121,6cyclictest9221-21/usr/sbin/munin21:10:210
14945993019,7cyclictest31691-21grep19:30:462
14945993018,7cyclictest4564-21processes19:45:372
1493399305,20cyclictest0-21swapper/020:15:360
1493399304,22cyclictest0-21swapper/021:21:330
1493399304,22cyclictest0-21swapper/019:40:230
1493399304,22cyclictest0-21swapper/019:12:480
1493399304,22cyclictest0-21swapper/019:08:120
1493399303,22cyclictest9228-21latency_hist20:00:040
1493399303,22cyclictest9228-21latency_hist20:00:040
14933993021,5cyclictest17420-21/usr/sbin/munin18:55:460
14933993011,15cyclictest0-21swapper/019:45:300
1494599295,20cyclictest0-21swapper/220:48:062
1494599294,21cyclictest0-21swapper/220:10:062
1494599294,20cyclictest0-21swapper/220:00:262
14945992911,13cyclictest0-21swapper/219:25:292
1493399294,20cyclictest0-21swapper/020:30:310
1494599284,20cyclictest0-21swapper/220:40:062
1494599284,20cyclictest0-21swapper/220:25:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional