ODA Questions

[youssef24eg]

hello , i am neubie to oracle oda and linux
and i received ine oda x5-2 and ininstructed to make it in virtualized environment
to hold prod and test vm
but i have many doubt for many things
1) is there difference between data and reco cause storage is spliiting between these two and  
is there a recommendation regarding vm and database where to put them
2) i couldn't find instruction to share vdisk between two VMs
3) for volume inside guest what is better primary  or lvm
for future modification and resizing

all material of oda doesn't have much explanation of technology
so is there good material for oda

[ErmanArslansOracleBlog]

1)Check the disks... Normally, redo will be on SSDs and Data will be on traditional disks.
read this: https://docs.oracle.com/cd/E22693_01/doc.12/e55580/referapp.htm

2)Check Oracle Database appliance command reference..

https://docs.oracle.com/cd/E68623_01/doc.121/e68637/GUID-A2C94E1D-CFAE-4CCC-A13D-04893D519DA7.htm#CMTAR512

oakcli create vdisk vdisk_name -repo repository_name -size size -type shared|local -sparse [-h]

-type shared | local

Sets the option of allowing the virtual disk to be shared by more than one virtual machine (shared) or used by only one virtual machine (local)

-> So, there is a type argument there.. So give it "shared" and try.

3) What do you mean by primary?
I use ext4 inside my VMs. But this is just because I like it.
LVM is easy to manage, so I recommend that one also.

4)I suggest you to read the ODA documentation..

Oracle Database Appliance Documentation Release 12.1.2.6

https://docs.oracle.com/cd/E68623_01/index.htm
(this release is 12.1.2.6, but you can find your release as well)

Go to -> https://docs.oracle.com/en/engineered-systems/
you will see ODA documentation there.. (for different releaes)

[youssef24eg]

TX for your fast reply ,  

i tried the shared disk between 2 vm and i mounted , but the data is not the same between both VM

when i create folder or file is not appearing into the second VM , so is there any configuration for linux for that ?

for oracle databases(ODA_Base) , i was instructed to provision only 1 TB for it from ASM

 is it possible ? cause all the storage provision inside DATA and RECO

by default where the ODA_Base store the databases?

On Thursday, December 22, 2016 11:32 AM, ErmanArslansOracleBlog [via Erman Arslan's Oracle Forum] <[hidden email]> wrote:

1)Check the disks... Normally, redo will be on SSDs and Data will be on traditional disks.
read this: https://docs.oracle.com/cd/E22693_01/doc.12/e55580/referapp.htm

2)Check Oracle Database appliance command reference..

https://docs.oracle.com/cd/E68623_01/doc.121/e68637/GUID-A2C94E1D-CFAE-4CCC-A13D-04893D519DA7.htm#CMTAR512

oakcli create vdisk vdisk_name -repo repository_name -size size -type shared|local -sparse [-h]

-type shared | local

Sets the option of allowing the virtual disk to be shared by more than one virtual machine (shared) or used by only one virtual machine (local)

-> So, there is a type argument there.. So give it "shared" and try.

3) What do you mean by primary?
I use ext4 inside my VMs. But this is just because I like it.
LVM is easy to manage, so I recommend that one also.

4)I suggest you to read the ODA documentation..

Oracle Database Appliance Documentation Release 12.1.2.6

https://docs.oracle.com/cd/E68623_01/index.htm
(this release is 12.1.2.6, but you can find your release as well)

Go to -> https://docs.oracle.com/en/engineered-systems/
you will see ODA documentation there.. (for different releaes)

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1798.html

To unsubscribe from ODA Questions, click here.
NAML

[ErmanArslansOracleBlog]

1)what filesystem are you using on those linuxvm guests?

2) when you create a disk on ODA/ on a shared repo , you are creating it on ASM actually. on ODA repos are on top of ASM.

22 Ara 2016 20:29 tarihinde "youssef24eg [via Erman Arslan's Oracle Forum]" <[hidden email]> yazdı:

TX for your fast reply ,  

i tried the shared disk between 2 vm and i mounted , but the data is not the same between both VM

when i create folder or file is not appearing into the second VM , so is there any configuration for linux for that ?

for oracle databases(ODA_Base) , i was instructed to provision only 1 TB for it from ASM

 is it possible ? cause all the storage provision inside DATA and RECO

by default where the ODA_Base store the databases?

On Thursday, December 22, 2016 11:32 AM, ErmanArslansOracleBlog [via Erman Arslan's Oracle Forum] <[hidden email]> wrote:

1)Check the disks... Normally, redo will be on SSDs and Data will be on traditional disks.
read this: https://docs.oracle.com/cd/E22693_01/doc.12/e55580/referapp.htm

2)Check Oracle Database appliance command reference..

https://docs.oracle.com/cd/E68623_01/doc.121/e68637/GUID-A2C94E1D-CFAE-4CCC-A13D-04893D519DA7.htm#CMTAR512

oakcli create vdisk vdisk_name -repo repository_name -size size -type shared|local -sparse [-h]

-type shared | local

Sets the option of allowing the virtual disk to be shared by more than one virtual machine (shared) or used by only one virtual machine (local)

-> So, there is a type argument there.. So give it "shared" and try.

3) What do you mean by primary?
I use ext4 inside my VMs. But this is just because I like it.
LVM is easy to manage, so I recommend that one also.

4)I suggest you to read the ODA documentation..

Oracle Database Appliance Documentation Release 12.1.2.6

https://docs.oracle.com/cd/E68623_01/index.htm
(this release is 12.1.2.6, but you can find your release as well)

Go to -> https://docs.oracle.com/en/engineered-systems/
you will see ODA documentation there.. (for different releaes)

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1798.html

To unsubscribe from ODA Questions, click here.
NAML

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1802.html

To start a new topic under Oracle Database Appliance(ODA) & VM, email [hidden email]
To unsubscribe from Erman Arslan's Oracle Forum, click here.
NAML

[youssef24eg]

i am using ext4 as a file system for this shared disk

On Friday, December 23, 2016 3:10 PM, ErmanArslansOracleBlog [via Erman Arslan's Oracle Forum] <[hidden email]> wrote:

1)what filesystem are you using on those linuxvm guests?

2) when you create a disk on ODA/ on a shared repo , you are creating it on ASM actually. on ODA repos are on top of ASM.

22 Ara 2016 20:29 tarihinde "youssef24eg [via Erman Arslan's Oracle Forum]" <[hidden email]> yazdı:

TX for your fast reply ,  

i tried the shared disk between 2 vm and i mounted , but the data is not the same between both VM

when i create folder or file is not appearing into the second VM , so is there any configuration for linux for that ?

for oracle databases(ODA_Base) , i was instructed to provision only 1 TB for it from ASM

 is it possible ? cause all the storage provision inside DATA and RECO

by default where the ODA_Base store the databases?

On Thursday, December 22, 2016 11:32 AM, ErmanArslansOracleBlog [via Erman Arslan's Oracle Forum] <[hidden email]> wrote:

1)Check the disks... Normally, redo will be on SSDs and Data will be on traditional disks.
read this: https://docs.oracle.com/cd/ E22693_01/doc.12/e55580/ referapp.htm

2)Check Oracle Database appliance command reference..

https://docs.oracle.com/cd/ E68623_01/doc.121/e68637/GUID- A2C94E1D-CFAE-4CCC-A13D- 04893D519DA7.htm#CMTAR512

oakcli create vdisk vdisk_name -repo repository_name -size size -type shared|local -sparse [-h]

-type shared | local

Sets the option of allowing the virtual disk to be shared by more than one virtual machine (shared) or used by only one virtual machine (local)

-> So, there is a type argument there.. So give it "shared" and try.

3) What do you mean by primary?
I use ext4 inside my VMs. But this is just because I like it.
LVM is easy to manage, so I recommend that one also.

4)I suggest you to read the ODA documentation..

Oracle Database Appliance Documentation Release 12.1.2.6

https://docs.oracle.com/cd/ E68623_01/index.htm
(this release is 12.1.2.6, but you can find your release as well)

Go to -> https://docs.oracle.com/en/ engineered-systems/
you will see ODA documentation there.. (for different releaes)

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle- forum.2340467.n4.nabble.com/ ODA-Questions-tp1796p1798.html

To unsubscribe from ODA Questions, click here.
NAML

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle- forum.2340467.n4.nabble.com/ ODA-Questions-tp1796p1802.html

To start a new topic under Oracle Database Appliance(ODA) & VM, email [hidden email]
To unsubscribe from Erman Arslan's Oracle Forum, click here.
NAML

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1803.html

To unsubscribe from ODA Questions, click here.
NAML

[ErmanArslansOracleBlog]

ext3, ext4 ... these kinds of filesystems are not suitable for sharing.

You need to use OCFS2 filesystem.

a ocfs2 file-system is a general-purpose, high-performance, high-availability, shared-disk file system intended for use in clusters.

see these docs:

https://docs.oracle.com/cd/E37670_01/E37355/html/ol_ocfs2.html#

https://docs.oracle.com/cd/E37670_01/E37355/html/ol_instcfg_ocfs2.html

[youssef24eg]

thanks so much , things now become much clearer

i also  find another way but i didn't try it yet

to create a volume through acfs then mount this vloume through NFS to both VMs

do u find any downside for both ways ? cause i think through acfs will be much easier 

what do u think about that ?

On Sunday, December 25, 2016 12:31 PM, ErmanArslansOracleBlog [via Erman Arslan's Oracle Forum] <[hidden email]> wrote:

ext3, ext4 ... these kinds of filesystems are not suitable for sharing.

You need to use OCFS2 filesystem.

a ocfs2 file-system is a general-purpose, high-performance, high-availability, shared-disk file system intended for use in clusters.

see these docs:

https://docs.oracle.com/cd/E37670_01/E37355/html/ol_ocfs2.html#

https://docs.oracle.com/cd/E37670_01/E37355/html/ol_instcfg_ocfs2.html

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1805.html

To unsubscribe from ODA Questions, click here.
NAML

[ErmanArslansOracleBlog]

so you won't use vdisks then.. You will use ACFS filesystems created on ODA BASE..  sounds good...

In ODA ACFS volumes (shared repositories) are exported to DOM 0 using NFS.. Instead, you will export ACFS volumes directly to the DOM U instead.

update me with the results.

[youssef24eg]

i tried to export Cloudfs which is already exists but i think i made some mistakes

1- vi /etc/export

exportfs /cloudfs 10.10.1.191(rw,sync,no_root_squash,fsid=1) 

exportfs -ra (when i issue this command all shared repo become unknown and all the oda base node 0 hang problem resolved only after reset host)

so how i can export cloudfs without affecting all other  shared repo ?

 

later i was planning to update /etc/fstab as below

echo "10.10.1.185:/cloudfs   /cloudfs    nfs     rw,intr,bg,hard,timeo=600,wsize=32768,rsize=32768,nfsvers=3,tcp,nolock,acregmin=0,acregmax=0 0 0"

note :10.10.1.185 is the virtual IP of node 0 for ODA_Base 

On Tuesday, December 27, 2016 4:40 PM, ErmanArslansOracleBlog [via Erman Arslan's Oracle Forum] <[hidden email]> wrote:

sounds good...

In ODA ACFS volumes (shared repositories) are exported to DOM 0 using NFS.. Instead, you will export ACFS volumes directly to the DOM U instead.

update me with the results.

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1811.html

To unsubscribe from ODA Questions, click here.
NAML

[ErmanArslansOracleBlog]

Send me the all node name-ip info. I need to understand the architecture that you want to build.. Please clearly explain the thing that you want to do.

2016-12-28 10:23 GMT+03:00 youssef24eg [via Erman Arslan's Oracle Forum] <[hidden email]>:

i tried to export Cloudfs which is already exists but i think i made some mistakes

1- vi /etc/export

exportfs /cloudfs 10.10.1.191(rw,sync,no_root_squash,fsid=1) 

exportfs -ra (when i issue this command all shared repo become unknown and all the oda base node 0 hang problem resolved only after reset host)

so how i can export cloudfs without affecting all other  shared repo ?

 

later i was planning to update /etc/fstab as below

echo "10.10.1.185:/cloudfs   /cloudfs    nfs     rw,intr,bg,hard,timeo=600,wsize=32768,rsize=32768,nfsvers=3,tcp,nolock,acregmin=0,acregmax=0 0 0"

note :10.10.1.185 is the virtual IP of node 0 for ODA_Base 

On Tuesday, December 27, 2016 4:40 PM, ErmanArslansOracleBlog [via Erman Arslan's Oracle Forum] <[hidden email]> wrote:

sounds good...

In ODA ACFS volumes (shared repositories) are exported to DOM 0 using NFS.. Instead, you will export ACFS volumes directly to the DOM U instead.

update me with the results.

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1811.html

To unsubscribe from ODA Questions, click here.
NAML

---

If you reply to this email, your message will be added to the discussion below:

http://erman-arslan-s-oracle-forum.2340467.n4.nabble.com/ODA-Questions-tp1796p1813.html

To start a new topic under Oracle Database Appliance(ODA) & VM, email [hidden email]
To unsubscribe from Erman Arslan's Oracle Forum, click here.
NAML

--

[ErmanArslansOracleBlog]

Bytheway;

Take a look at "http://docs.oracle.com/cd/E56825_01/doc.121/e56823.pdf"
Oracle® Database Appliance Kit for WebLogic Server
Release Notes
Release 12.1.3.0.0


Section: Configuring a Common NFS Mount Point to be Shared Across all Coherence VMs..

It uses "exportfs -a"

[ErmanArslansOracleBlog]

Also look at this blog post:
http://ermanarslan.blogspot.com.tr/2015/11/oda-x4-virtualized-adding-acfs-mount.html

In this blog post, I add the acfsmount-nfs info to the /var/lib/nfs/etab and execute "exportfs" command.
Afterwards, I mount the exported ACFS using nfs from another machine.

[root@ermansrv1 ~]#vi /var/lib/nfs/etab

/u01/app/acfsmounts/ssdrepo1    192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)
/u01/app/sharedrepo/vmtemp2     192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)
/u01/app/sharedrepo/vmrepo1     192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)
/u01/app/sharedrepo/vmtemp1     192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)  (ADDED an export for the Hypervisors private network ip)

[root@ermansrv1 ~]# exportfs

root@ermanhype ~]#mount -t nfs 192.168.16.17:/u01/app/acfsmounts/ssdrepo1 /OVS/ssdrepo1   (The ip address is the private network address of ODA BASE machine 1)

[youssef24eg]

i am sorry i am bothering you but you are very experience in ODA and in Oracle
what i want to achieve is to have two application tier EBS 12.2 in production  HA and the DBA ask for shared storage for these two VMs
but we want all the setup of oracle inside the ODA so we don't have external NFS to provide to him
so that why i was trying to check this if it is possible to do it , oracle said it is not certified in ODA , so do you think that is possible to implement it in prod env. and on ODA X5-2 with infini band interconnect
cause i am reading many limitation for this also

IP ADDR DOM0 Node0 10.10.1.181
IP ADDR DOM0 Node1 10.10.1.182


ODA BASE  Node 0 10.10.183
ODA BASE Node 1 10.10.1.184
ODA BASE VIP node 0 10.10.1.185
ODA BASE VIP node 1 10.10.1.186

APP 1 10.10.1.191
APP 2 10.10.1.192


i was thinking to export the shared vloume and mount it by VIP not by the priv IP

but why node 0 shared repo hang when i ran the command exportfs  -ra ?  btw all the machines was running

[ErmanArslansOracleBlog]

"Not certified" doesn't mean that "it is not supported"
"Not certified" doesn't mean that "it can not be done"

So , this can be done.
I mean "creating an ACFS filesystem on ODA Base and exporting that filesystem as NFS share to the app tier vms", should work.

Did you see my previous reply?

-->

Also look at this blog post:
http://ermanarslan.blogspot.com.tr/2015/11/oda-x4-virtualized-adding-acfs-mount.html

In this blog post, I add the acfsmount-nfs info to the /var/lib/nfs/etab and execute "exportfs" command.
Afterwards, I mount the exported ACFS using nfs from another machine.

[root@ermansrv1 ~]#vi /var/lib/nfs/etab

/u01/app/acfsmounts/ssdrepo1    192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)
/u01/app/sharedrepo/vmtemp2     192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)
/u01/app/sharedrepo/vmrepo1     192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)
/u01/app/sharedrepo/vmtemp1     192.168.16.14(rw,sync,no_wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,mapping=identity,anonuid=65534,anongid=65534)  (ADDED an export for the Hypervisors private network ip)

[root@ermansrv1 ~]# exportfs

root@ermanhype ~]#mount -t nfs 192.168.16.17:/u01/app/acfsmounts/ssdrepo1 /OVS/ssdrepo1   (The ip address is the private network address of ODA BASE machine 1)

[Youssef]

ur blog is better than oracle pdf , i followed ur steps and now it is working perfectly
only one issue after reboot it is not mounted ,
i enabled nfs service on startup and edit etc/fstab , so what is missing ?
last question , i read that nfslock is not supported  so
if i use this shared disk for both app tier it will not get corrupted ?

[ErmanArslansOracleBlog]

1)
first of all;

For building a Shared APPL_TOP, you could export a local filesystem from your App VM node 1 to node 2 and mount it from node2 using NFS.
That is you could give a vdisk to App VM node 1 and export it to Apps Node 2 using NFS.
It is explained Doc ID 1375769.1 with all its details.


On the other hand, what you are trying to do is something different. You want to export an ACFS from ODA nodes to App VM Node 1 and Node 2 at the same time.
I think you are not an Apps DBA and that's why you are only dealing with the Shared disk and NFS part.
So, your Apps DBA told you to give him a shared disk and you are trying to implement his/or her wish.

Well.
I think it is going to work, because it is documented in "Oracle Database Appliance Kit for WebLogic Server"
("http://docs.oracle.com/cd/E56825_01/doc.121/e56823.pdf") --> section: Configuring a Common NFS Mount Point to be Shared Across all Coherence VMs


While this note is not for EBS, there is something important written in 1375769.1.

"When entering the location of the install directories for the application tier install, you can either enter the location of a local drive on the machine, or an NFS location that is accessible from the primary application tier node. The recommendation is to install the application tier file system on to a remote file system that can be mounted across various application tier nodes."

Moreoever what is written there is -> "The recommendation is to install the application tier file system on to a remote file system that can be mounted across various application tier nodes."

So there is nothing wrong with your approach.


nfslock should be on in your ODA Base nodes..
Check it using chkconfig..
For shared appl_Top (using NFS), read the document named “Sharing the Application Tier File System in Oracle E-Business Suite Release 12.2” (Doc ID 1375769.1).
There are some recommended NFS parameters and you will find them in section : 4.1 Export the Required Application Tier File System From the Primary Node


2)
About fstab boot automount problem..
There is an easy workaround: you can write the mount command to /etc/rc.d/rc.local.

However, it is better to solve your problem .

your App vms which should have auto mount the nfs shares during boot; are Linux ,right?

I need to see your mount options and enabled services to support those..
so in order to have the required info, please send me the following outputs;


-cat /etc/fstab
-mount -a
-chkconfig --list
-/var/log/messages
-runlevel
-cat /etc/rc.conf

[youssef24eg]

i had one character wrong in /etc/fstab and now it is mounting fine after reboot
but i tried to reboot this node but after reboot the changes into /var/lib/nfs/etab was cleared
and the the folder is not exported any more, so how to make this permanent ?

also when i reboot one node , it is hanging and not rebooting
only powercycle from ILOM make it reboot
is there any procedure for simple reboot ?

[ErmanArslansOracleBlog]

1)Write it to /etc/exports file, and retry...
2)Check what is hanging.. Check in the process level.
--check OS logs. Check NFS server, ensure it can be stopped and ensure the NFS mount points can be unmounted.. (maybe it is waiting for an open NFS operation?) -- unmount your nfs shares from your apps nodes, before rebooting the ODA nodes.

[youssef24eg]

thanks so much.

[ErmanArslansOracleBlog]

Hi,

Update me with the outcome.