You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-10-01 - 00:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2s.osadl.org (updated Sat Sep 30, 2023 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7865702309304,3sleep20-21swapper/207:07:352
7865152307302,3sleep30-21swapper/307:06:463
7866982301296,3sleep70-21swapper/707:09:217
7866712299294,3sleep60-21swapper/607:09:006
7865952299294,3sleep10-21swapper/107:07:541
7866992296291,3sleep80-21swapper/807:09:228
7866432294288,4sleep50-21swapper/507:08:385
7865722294289,3sleep40-21swapper/407:07:374
7864402293287,4sleep90-21swapper/907:05:429
7864872290284,4sleep00-21swapper/007:06:230
78702099210,0cyclictest0-21swapper/809:30:238
78702099210,0cyclictest0-21swapper/807:10:278
78702399200,20cyclictest0-21swapper/910:55:279
78702399200,20cyclictest0-21swapper/907:10:249
787020992019,1cyclictest0-21swapper/808:00:258
787020992018,2cyclictest0-21swapper/811:00:298
78702099200,0cyclictest0-21swapper/810:20:248
78699599201,19cyclictest788319-21grep07:10:212
410612999200,0ntpd0-21swapper/610:35:066
787023991919,0cyclictest0-21swapper/911:50:309
787023991919,0cyclictest0-21swapper/910:50:289
787023991919,0cyclictest0-21swapper/909:35:229
78702399190,19cyclictest0-21swapper/909:50:259
78702399190,19cyclictest0-21swapper/908:50:269
78702399190,19cyclictest0-21swapper/908:46:099
78702399190,19cyclictest0-21swapper/908:35:249
78702399190,19cyclictest0-21swapper/908:15:239
787020991919,0cyclictest0-21swapper/810:30:288
787020991919,0cyclictest0-21swapper/808:15:248
787020991919,0cyclictest0-21swapper/807:50:248
787020991919,0cyclictest0-21swapper/807:25:258
78702099190,19cyclictest0-21swapper/811:45:288
78701199190,19cyclictest850210-21ntpq09:05:246
78700399190,1cyclictest919212-21ntpq11:15:284
78700399190,1cyclictest829140-21smart_nvme0n108:25:274
78700399190,1cyclictest800587-40dnf07:32:304
78700399190,19cyclictest868744-21ntpq09:40:244
78700399190,19cyclictest852268-21diskstats09:10:204
78699999190,1cyclictest871415-21ntpq09:45:243
78699999190,19cyclictest879375-21ldconfig10:00:273
786995991918,1cyclictest876705-21ntpq09:55:242
78698899190,1cyclictest935155-21ntp_states11:45:290
787023991818,0cyclictest0-21swapper/910:30:269
787023991818,0cyclictest0-21swapper/910:25:309
787023991818,0cyclictest0-21swapper/909:05:259
787023991818,0cyclictest0-21swapper/907:20:229
787023991818,0cyclictest0-21swapper/907:20:229
787023991818,0cyclictest0-21swapper/907:15:239
787023991818,0cyclictest0-21swapper/907:15:239
78702399180,18cyclictest0-21swapper/909:25:259
78702399180,18cyclictest0-21swapper/907:25:249
78702399180,0cyclictest0-21swapper/907:40:249
787020991818,0cyclictest0-21swapper/808:55:258
787020991818,0cyclictest0-21swapper/808:45:198
787020991817,1cyclictest913886-21ntpq11:05:318
787020991817,1cyclictest0-21swapper/812:35:218
787020991817,1cyclictest0-21swapper/811:25:258
787020991817,1cyclictest0-21swapper/810:05:248
78702099180,18cyclictest0-21swapper/811:50:318
78702099180,18cyclictest0-21swapper/810:00:278
78702099180,18cyclictest0-21swapper/808:25:268
78702099180,0cyclictest0-21swapper/807:15:008
78702099180,0cyclictest0-21swapper/807:15:008
787016991818,0cyclictest0-21swapper/709:50:277
787016991818,0cyclictest0-21swapper/708:30:237
787016991818,0cyclictest0-21swapper/708:25:247
787016991818,0cyclictest0-21swapper/707:10:187
78701699180,18cyclictest863443-21ntpq09:30:237
78701699180,18cyclictest0-21swapper/711:50:327
78701699180,18cyclictest0-21swapper/711:25:267
78701699180,18cyclictest0-21swapper/709:55:247
78701699180,18cyclictest0-21swapper/709:20:237
78701699180,18cyclictest0-21swapper/708:55:267
78701699180,18cyclictest0-21swapper/707:25:257
78701699180,18cyclictest0-21swapper/707:15:007
78701699180,18cyclictest0-21swapper/707:15:007
78701699180,0cyclictest0-21swapper/712:35:217
78701699180,0cyclictest0-21swapper/708:45:207
78701699180,0cyclictest0-21swapper/708:00:257
787011991817,1cyclictest905932-21awk10:50:286
787011991817,1cyclictest788503-21ntpq07:10:226
78701199180,18cyclictest0-21swapper/609:45:236
78701199180,18cyclictest0-21swapper/609:35:246
78701199180,18cyclictest0-21swapper/608:40:286
78701199180,18cyclictest0-21swapper/607:25:256
78701199180,18cyclictest0-21swapper/607:20:226
78701199180,18cyclictest0-21swapper/607:20:226
78701199180,0cyclictest0-21swapper/611:40:296
78701199180,0cyclictest0-21swapper/608:55:256
787007991818,0cyclictest0-21swapper/510:25:315
787007991818,0cyclictest0-21swapper/509:50:255
78700799180,18cyclictest823664-21ntpq08:15:245
78700799180,18cyclictest0-21swapper/509:35:245
78700799180,18cyclictest0-21swapper/509:30:245
78700799180,18cyclictest0-21swapper/507:10:255
78700799180,0cyclictest0-21swapper/508:10:255
787003991818,0cyclictest0-21swapper/409:25:234
787003991817,1cyclictest945766-21ntp_states12:05:254
787003991817,1cyclictest905952-21ntp_states10:50:284
78700399180,18cyclictest956351-21ntpq12:25:254
78700399180,18cyclictest895326-21ntpq10:30:284
78700399180,18cyclictest879418-21ntp_states10:00:284
78700399180,18cyclictest831652-21ntpq08:30:244
78700399180,18cyclictest0-21swapper/412:15:264
78700399180,18cyclictest0-21swapper/409:00:254
78700399180,18cyclictest0-21swapper/408:45:234
78700399180,18cyclictest0-21swapper/407:10:224
78700399180,0cyclictest826320-21ntpq08:20:234
78700399180,0cyclictest0-21swapper/410:20:244
78700399180,0cyclictest0-21swapper/407:55:254
78699999180,18cyclictest895294-21ntpq10:30:273
78699999180,18cyclictest852892-21ntpq09:10:243
78699999180,18cyclictest0-21swapper/308:15:243
78699999180,0cyclictest0-21swapper/310:55:263
78699999180,0cyclictest0-21swapper/307:10:563
786995991817,1cyclictest940399-21ntpq11:55:252
786995991817,1cyclictest916723-21smart_nvme0n111:10:322
78699599180,1cyclictest908570-21ntpq10:55:282
78699599180,1cyclictest863497-21ntpq09:30:242
78699599180,1cyclictest840293-40dnf08:46:092
78699599180,18cyclictest871457-21ntp_states09:45:252
78699599180,18cyclictest0-21swapper/211:45:292
78699599180,18cyclictest0-21swapper/210:20:242
78699299180,18cyclictest940458-21ntpq11:55:271
78698899180,18cyclictest794476-21ntpq07:20:210
78698899180,18cyclictest794476-21ntpq07:20:210
787023991717,0cyclictest887286-21strings10:15:229
787023991717,0cyclictest0-21swapper/911:40:299
787023991717,0cyclictest0-21swapper/911:10:329
787023991717,0cyclictest0-21swapper/911:05:319
787023991717,0cyclictest0-21swapper/910:00:239
787023991717,0cyclictest0-21swapper/909:40:259
787023991717,0cyclictest0-21swapper/909:30:239
787023991717,0cyclictest0-21swapper/909:10:249
787023991717,0cyclictest0-21swapper/909:00:279
787020991717,0cyclictest0-21swapper/811:10:288
787020991717,0cyclictest0-21swapper/810:15:238
787020991717,0cyclictest0-21swapper/809:45:248
787020991717,0cyclictest0-21swapper/809:10:248
787020991717,0cyclictest0-21swapper/808:50:258
787020991717,0cyclictest0-21swapper/808:35:248
787020991717,0cyclictest0-21swapper/808:30:238
787020991716,1cyclictest0-21swapper/812:15:248
787020991716,1cyclictest0-21swapper/809:50:248
787020991715,2cyclictest1080-21snmpd07:20:588
787020991715,2cyclictest1080-21snmpd07:20:588
78702099170,17cyclictest0-21swapper/811:35:238
78702099170,0cyclictest0-21swapper/809:35:248
787016991717,0cyclictest0-21swapper/712:25:247
787016991717,0cyclictest0-21swapper/712:05:257
787016991717,0cyclictest0-21swapper/712:00:247
787016991717,0cyclictest0-21swapper/711:45:267
787016991717,0cyclictest0-21swapper/710:55:277
787016991717,0cyclictest0-21swapper/710:00:277
787016991717,0cyclictest0-21swapper/708:50:267
787016991717,0cyclictest0-21swapper/708:15:237
787016991717,0cyclictest0-21swapper/708:10:257
787016991717,0cyclictest0-21swapper/708:05:217
787016991717,0cyclictest0-21swapper/707:50:247
787011991717,0cyclictest0-21swapper/610:55:276
787011991717,0cyclictest0-21swapper/610:25:306
787011991717,0cyclictest0-21swapper/610:15:236
787011991717,0cyclictest0-21swapper/609:50:256
787011991717,0cyclictest0-21swapper/609:25:256
787011991717,0cyclictest0-21swapper/608:50:266
787011991717,0cyclictest0-21swapper/608:46:096
787011991717,0cyclictest0-21swapper/608:35:246
787011991717,0cyclictest0-21swapper/608:15:236
787011991717,0cyclictest0-21swapper/607:40:246
78701199170,17cyclictest0-21swapper/611:05:326
78701199170,17cyclictest0-21swapper/608:10:256
78701199170,0cyclictest0-21swapper/609:10:236
78701199170,0cyclictest0-21swapper/607:15:246
78701199170,0cyclictest0-21swapper/607:15:246
787007991717,0cyclictest0-21swapper/512:00:215
787007991717,0cyclictest0-21swapper/511:45:275
787007991717,0cyclictest0-21swapper/511:40:295
787007991717,0cyclictest0-21swapper/511:05:325
787007991717,0cyclictest0-21swapper/511:00:285
787007991717,0cyclictest0-21swapper/509:40:255
787007991717,0cyclictest0-21swapper/508:50:265
787007991717,0cyclictest0-21swapper/508:45:195
787007991717,0cyclictest0-21swapper/507:15:005
787007991717,0cyclictest0-21swapper/507:15:005
78700799170,0cyclictest0-21swapper/510:30:265
787003991717,0cyclictest0-21swapper/411:50:324
787003991717,0cyclictest0-21swapper/411:45:294
787003991717,0cyclictest0-21swapper/410:10:254
787003991717,0cyclictest0-21swapper/409:30:244
787003991717,0cyclictest0-21swapper/408:00:254
787003991717,0cyclictest0-21swapper/407:35:244
787003991717,0cyclictest0-21swapper/407:25:274
787003991716,1cyclictest961307-21irqstats12:35:244
787003991716,1cyclictest943059-21ntpq12:00:254
787003991716,1cyclictest929218-21diskstats11:35:194
78700399170,17cyclictest0-21swapper/411:10:304
78700399170,17cyclictest0-21swapper/408:40:274
786999991717,0cyclictest0-21swapper/311:25:253
786999991717,0cyclictest0-21swapper/311:00:303
786999991717,0cyclictest0-21swapper/309:50:273
786999991717,0cyclictest0-21swapper/309:35:243
786999991717,0cyclictest0-21swapper/308:45:203
786999991717,0cyclictest0-21swapper/307:25:243
786999991716,1cyclictest932473-21ntpq11:40:303
78699999170,17cyclictest0-21swapper/311:05:313
78699999170,16cyclictest0-21swapper/312:35:213
78699999170,0cyclictest0-21swapper/310:35:303
78699999170,0cyclictest0-21swapper/309:30:233
78699999170,0cyclictest0-21swapper/308:10:253
786995991717,0cyclictest0-21swapper/211:25:272
786995991717,0cyclictest0-21swapper/211:15:282
786995991717,0cyclictest0-21swapper/210:25:332
786995991717,0cyclictest0-21swapper/208:05:212
786995991717,0cyclictest0-21swapper/207:35:232
786995991716,1cyclictest826345-21ntpq08:20:232
78699599170,17cyclictest0-21swapper/212:25:262
78699599170,17cyclictest0-21swapper/212:05:242
78699599170,17cyclictest0-21swapper/208:25:252
78699599170,17cyclictest0-21swapper/208:00:252
78699599170,16cyclictest882036-21ntpq10:05:252
78699599170,0cyclictest0-21swapper/212:00:262
78699599170,0cyclictest0-21swapper/210:50:272
786992991717,0cyclictest0-21swapper/110:25:301
78699299170,17cyclictest0-21swapper/110:15:221
78699299170,17cyclictest0-21swapper/110:00:291
78699299170,0cyclictest0-21swapper/107:55:251
78699299170,0cyclictest0-21swapper/107:20:221
78699299170,0cyclictest0-21swapper/107:20:221
78699299170,0cyclictest0-21swapper/107:10:231
786988991716,1cyclictest916578-21ntp_states11:10:300
786988991716,1cyclictest900640-21ntpq10:40:370
786988991716,1cyclictest876718-21ntpq09:55:240
786988991716,1cyclictest860795-21ntpq09:25:230
78698899170,1cyclictest884711-21ntpq10:10:250
78698899170,17cyclictest0-21swapper/012:25:250
78698899170,17cyclictest0-21swapper/007:10:550
410612999170,0ntpd0-21swapper/111:35:061
787023991616,0cyclictest0-21swapper/910:35:289
787023991616,0cyclictest0-21swapper/910:07:269
787023991616,0cyclictest0-21swapper/908:10:259
787023991616,0cyclictest0-21swapper/907:55:269
787020991616,0cyclictest0-21swapper/810:45:368
787020991616,0cyclictest0-21swapper/807:45:258
787020991615,1cyclictest0-21swapper/811:40:298
787016991616,0cyclictest0-21swapper/712:00:007
787016991616,0cyclictest0-21swapper/711:05:317
787016991616,0cyclictest0-21swapper/710:20:247
787016991616,0cyclictest0-21swapper/710:05:247
787016991616,0cyclictest0-21swapper/709:45:247
787016991616,0cyclictest0-21swapper/707:30:257
787011991616,0cyclictest0-21swapper/611:50:306
787011991615,1cyclictest1080-21snmpd12:22:296
787007991616,0cyclictest958941-21strings12:30:255
787007991616,0cyclictest0-21swapper/511:25:265
787007991616,0cyclictest0-21swapper/511:15:275
787007991616,0cyclictest0-21swapper/511:10:285
787007991616,0cyclictest0-21swapper/510:45:415
787007991616,0cyclictest0-21swapper/509:45:245
787007991616,0cyclictest0-21swapper/508:55:255
787007991615,1cyclictest1080-21snmpd12:15:325
787007991615,1cyclictest1080-21snmpd10:23:405
787007991615,1cyclictest1080-21snmpd07:49:505
787003991616,0cyclictest0-21swapper/411:55:264
787003991616,0cyclictest0-21swapper/410:55:294
787003991616,0cyclictest0-21swapper/409:20:254
787003991616,0cyclictest0-21swapper/407:40:224
787003991615,1cyclictest1080-21snmpd08:52:464
786999991616,0cyclictest892696-21ntpq10:25:313
786999991616,0cyclictest0-21swapper/312:25:233
786999991616,0cyclictest0-21swapper/312:00:213
786999991616,0cyclictest0-21swapper/308:55:263
786999991616,0cyclictest0-21swapper/307:15:233
786999991616,0cyclictest0-21swapper/307:15:233
786999991615,1cyclictest919185-21ntpq11:15:273
786999991615,1cyclictest1080-21snmpd09:08:263
786999991615,1cyclictest1080-21snmpd07:50:233
786999991615,1cyclictest1080-21snmpd07:39:323
786995991616,0cyclictest951055-21ntpq12:15:262
786995991616,0cyclictest0-21swapper/211:50:312
786995991616,0cyclictest0-21swapper/211:00:292
786995991616,0cyclictest0-21swapper/210:00:282
786995991616,0cyclictest0-21swapper/209:50:282
786995991616,0cyclictest0-21swapper/207:32:302
786995991615,1cyclictest844897-21ntpq08:55:262
786992991616,0cyclictest921890-21ntpq11:20:331
786992991616,0cyclictest0-21swapper/110:50:281
786992991616,0cyclictest0-21swapper/110:10:251
786992991616,0cyclictest0-21swapper/109:35:231
786992991616,0cyclictest0-21swapper/109:00:251
786992991616,0cyclictest0-21swapper/107:15:241
786992991616,0cyclictest0-21swapper/107:15:241
786992991615,1cyclictest1080-21snmpd11:05:571
78699299160,0cyclictest0-21swapper/108:40:271
78699299160,0cyclictest0-21swapper/107:40:241
786988991616,0cyclictest802474-21ntpq07:35:250
786988991616,0cyclictest0-21swapper/011:55:250
786988991616,0cyclictest0-21swapper/010:15:220
786988991616,0cyclictest0-21swapper/009:45:240
786988991616,0cyclictest0-21swapper/007:32:300
786988991615,1cyclictest882084-21ntpq10:05:260
786988991615,1cyclictest847570-21ntpq09:00:250
786988991615,1cyclictest1080-21snmpd11:34:270
78698899160,16cyclictest0-21swapper/012:05:250
78698899160,16cyclictest0-21swapper/008:40:270
78698899160,16cyclictest0-21swapper/007:40:230
78698899160,16cyclictest0-21swapper/007:15:200
78698899160,16cyclictest0-21swapper/007:15:200
787023991515,0cyclictest0-21swapper/912:00:269
787023991515,0cyclictest0-21swapper/908:40:289
787023991514,1cyclictest1080-21snmpd12:28:119
787023991514,1cyclictest1080-21snmpd11:24:429
787023991514,1cyclictest1080-21snmpd07:34:029
787020991515,0cyclictest0-21swapper/810:55:288
787020991515,0cyclictest0-21swapper/810:40:308
787020991515,0cyclictest0-21swapper/810:35:298
787020991515,0cyclictest0-21swapper/808:05:218
787020991514,1cyclictest1080-21snmpd12:28:358
787016991515,0cyclictest0-21swapper/711:20:287
787016991515,0cyclictest0-21swapper/709:35:247
787011991515,0cyclictest0-21swapper/610:30:266
787011991515,0cyclictest0-21swapper/609:40:256
787011991515,0cyclictest0-21swapper/609:30:236
787011991515,0cyclictest0-21swapper/609:00:276
787007991515,0cyclictest0-21swapper/509:00:245
787007991515,0cyclictest0-21swapper/507:20:225
787007991515,0cyclictest0-21swapper/507:20:225
787007991514,1cyclictest1080-21snmpd11:54:465
787007991514,1cyclictest1080-21snmpd10:03:075
787007991514,1cyclictest1080-21snmpd09:14:295
787003991515,0cyclictest2697250irq/132-eno111:22:394
787003991515,0cyclictest0-21swapper/410:15:224
787003991515,0cyclictest0-21swapper/408:05:224
787003991514,1cyclictest1080-21snmpd08:55:464
786999991515,0cyclictest0-21swapper/311:50:313
786999991515,0cyclictest0-21swapper/311:45:273
786999991515,0cyclictest0-21swapper/311:10:283
786999991515,0cyclictest0-21swapper/309:40:253
786999991515,0cyclictest0-21swapper/308:50:263
786995991515,0cyclictest836915-21ntpq08:40:272
786995991515,0cyclictest0-21swapper/210:30:282
786995991515,0cyclictest0-21swapper/209:10:262
786995991515,0cyclictest0-21swapper/207:50:232
786995991514,1cyclictest797138-21ntpq07:25:252
786995991514,1cyclictest1080-21snmpd11:06:092
786992991515,0cyclictest0-21swapper/111:40:301
786992991515,0cyclictest0-21swapper/110:05:261
786992991515,0cyclictest0-21swapper/109:50:301
786992991515,0cyclictest0-21swapper/109:40:251
786992991515,0cyclictest0-21swapper/109:25:251
786992991515,0cyclictest0-21swapper/108:35:241
786992991514,1cyclictest799806-21ntpq07:30:261
786992991514,1cyclictest1080-21snmpd12:11:581
786988991515,0cyclictest905928-21ntpq10:50:280
786988991515,0cyclictest1080-21snmpd11:35:540
786988991515,0cyclictest0-21swapper/012:20:240
786988991515,0cyclictest0-21swapper/011:15:280
786988991515,0cyclictest0-21swapper/010:20:240
786988991515,0cyclictest0-21swapper/009:50:280
786988991515,0cyclictest0-21swapper/009:20:250
786988991515,0cyclictest0-21swapper/009:15:240
786988991515,0cyclictest0-21swapper/008:45:220
786988991514,1cyclictest898949-21sadc10:39:590
786988991514,1cyclictest815702-21ntpq08:00:250
786988991514,1cyclictest1080-21snmpd12:10:580
787023991414,0cyclictest0-21swapper/910:10:249
787023991414,0cyclictest0-21swapper/908:55:269
787023991413,1cyclictest1080-21snmpd10:41:539
787016991414,0cyclictest0-21swapper/710:50:267
787016991414,0cyclictest0-21swapper/709:10:247
787011991414,0cyclictest0-21swapper/611:10:326
787011991414,0cyclictest0-21swapper/610:00:236
787007991414,0cyclictest0-21swapper/510:50:285
787007991414,0cyclictest0-21swapper/510:35:295
787007991414,0cyclictest0-21swapper/510:10:265
787007991414,0cyclictest0-21swapper/510:07:275
787007991414,0cyclictest0-21swapper/508:05:215
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional