Softlayer network speed test

Test network latency, upload/download rate to AMS01 data center.

http://speedtest.ams01.softlayer.com

Only half of CPUs are using?

During test, only half of CPUs are using, and the rest of CPUs will be picked when the current using CPU are over 50% busy. Why? CPU affinity?

-----------------------------?

1. CPU affinity?

2. CFS scheduler? 

--------------------------
when do application deployment, the CPU usage leverage all CPUs, so it  won't be CPU affinity.

CPU goes down

It appears the CPU usage decreasing, and all response times also go down, but there is no effect on the overall throughput. What will causes CPU decreasing?

-- The reason is the run-time data was decreasing.

Weblogic 12c datasource odd behavior

Created a weblogic cluster with 2 members, and created two db datasources for the cluster.  While start both servers, there is always one datasource can't be found on a server. Error log as below:

####<Jan 6, 2015 8:34:19 PM PST> <Info> <JDBC> <wlserver> <weblogicserver> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1420605259151> <BEA-001177> <Creating data source connection pool named ORADS, URL = jdbc:oracle:thin:@oracledb:1521:dbname, Properties = user=gcduser;databaseName=dbname;serverName=oracledb;portnumber=1521;.>
####<Jan 6, 2015 8:34:19 PM PST> <Info> <Common> <wlserver> <weblogicserver> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1420605259792> <BEA-000626> <Free resources in pool "ORADS" will be tested every "120" seconds.>
####<Jan 6, 2015 8:36:46 PM PST> <Info> <JDBC> <wlserver> <weblogicserver> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1420605406141> <BEA-001508> <Destroying data source connection pool ORADS.>
####<Jan 6, 2015 8:36:46 PM PST> <Info> <JDBC> <wlserver> <weblogicserver> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1420605406141> <BEA-001155> <The following exception has occurred:

weblogic.common.ResourceException: Unknown Data Source ORADS
at weblogic.jdbc.common.internal.ConnectionPoolManager.shutdownConnectionPool(ConnectionPoolManager.java:630)
at weblogic.jdbc.common.internal.ConnectionPoolManager.shutdownAndDestroyPool(ConnectionPoolManager.java:602)
at weblogic.jdbc.common.internal.ConnectionPoolManager.shutdownAndDestroyPool(ConnectionPoolManager.java:533)
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:330)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:100)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:172)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:167)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.flow.ModuleStateDriver.prepare(ModuleStateDriver.java:38)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:139)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:55)
at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:706)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:237)
at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:158)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:66)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:39)
at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191)
at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:21)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:165)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:186)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:94)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
>
####<Jan 6, 2015 8:36:46 PM PST> <Error> <Deployer> <wlserver> <weblogicserver> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1420605406144> <BEA-149205> <Failed to initialize the application "ORADS" due to error weblogic.application.ModuleException:
weblogic.application.ModuleException:
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:337)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:100)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:172)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:167)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.flow.ModuleStateDriver.prepare(ModuleStateDriver.java:38)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:139)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:55)
at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:706)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:237)
at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:158)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:66)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:39)
at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191)
at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:21)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:165)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:186)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:94)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: weblogic.common.ResourceException: weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Connection reset
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:288)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1310)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1227)
at weblogic.common.resourcepool.ResourcePoolImpl.start(ResourcePoolImpl.java:250)
at weblogic.jdbc.common.internal.ConnectionPool.doStart(ConnectionPool.java:1396)
at weblogic.jdbc.common.internal.ConnectionPool.start(ConnectionPool.java:187)
at weblogic.jdbc.common.internal.ConnectionPoolManager.createAndStartPool(ConnectionPoolManager.java:520)
at weblogic.jdbc.common.internal.ConnectionPoolManager.createAndStartPool(ConnectionPoolManager.java:442)
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:288)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:100)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:172)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:167)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.flow.ModuleStateDriver.prepare(ModuleStateDriver.java:38)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:139)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:55)
at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:706)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:237)
at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:158)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:66)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:39)
at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191)
at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:21)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:165)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:186)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:94)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

Solution :
1. re-create two node (the original 2nd node was created by copy node-1 by template pack/unpacking)
2. start server by running command : ./startManagedWebLogic.sh Server2 domainServer:7001 (originally there was no domainServer:7001 appending)

weblogic 12c start server slowly

update domain/bin/setDomainEnv.sh
add :

JAVA_OPTIONS="-Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTIONS